Oct 31 00:53:08.070559 ip-10-0-140-244 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 31 00:53:08.070569 ip-10-0-140-244 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 31 00:53:08.070576 ip-10-0-140-244 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 31 00:53:08.070826 ip-10-0-140-244 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 31 00:53:18.125754 ip-10-0-140-244 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 31 00:53:18.125779 ip-10-0-140-244 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9d078ce9753b4550a2c136c362e2393d --
Oct 31 00:55:25.079242 ip-10-0-140-244 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 00:55:25.458905 ip-10-0-140-244 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:55:25.458905 ip-10-0-140-244 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 00:55:25.458905 ip-10-0-140-244 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:55:25.458905 ip-10-0-140-244 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:55:25.464049 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.463970    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.464049 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464048    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464052    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464055    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464059    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464062    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464065    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464068    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464071    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464074    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464076    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464079    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464082    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464084    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464087    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464090    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464092    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464095    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464097    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464100    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464105    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.464154 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464109    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464112    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464115    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464117    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464121    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464126    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464134    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464138    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464141    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464144    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464146    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464149    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464151    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464154    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464157    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464160    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464163    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464167    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464170    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.464614 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464172    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464175    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464177    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464180    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464182    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464185    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464188    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464190    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464193    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464195    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464198    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464201    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464204    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464206    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464208    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464211    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464214    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464216    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464218    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464221    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.465079 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464224    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464227    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464229    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464232    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464234    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464237    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464239    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464242    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464245    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464247    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464250    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464252    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464255    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464257    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464260    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464262    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464265    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464267    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464270    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.465545 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464272    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464275    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464278    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464281    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464284    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464287    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464289    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464292    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464295    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464297    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464301    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464303    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464306    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464308    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464311    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464314    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464316    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464318    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464321    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.466008 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464324    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464327    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464330    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464332    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464335    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464338    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464340    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.464343    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465112    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465119    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465122    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465125    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465128    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465130    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465133    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465135    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465138    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465141    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465144    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.466457 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465147    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465150    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465153    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465155    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465158    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465161    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465164    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465166    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465169    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465171    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465174    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465176    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465179    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465182    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465185    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465189    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465193    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465196    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465199    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.466965 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465201    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465204    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465206    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465209    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465211    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465214    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465216    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465219    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465222    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465224    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465227    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465229    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465232    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465234    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465237    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465240    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465243    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465246    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465249    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465252    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.467410 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465254    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465257    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465259    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465262    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465265    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465267    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465271    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465273    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465276    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465279    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465282    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465286    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465289    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465292    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465294    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465297    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465299    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465302    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465304    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465307    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.467969 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465309    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465312    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465314    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465317    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465319    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465322    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465324    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465328    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465331    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465334    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465336    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465339    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465342    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465344    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465347    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465349    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465352    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465355    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465357    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465360    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.468445 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465363    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465365    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465368    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465370    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465373    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465375    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465378    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465380    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465383    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465385    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465388    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465391    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465393    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465396    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465398    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465401    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465887    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465894    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465898    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.468952 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465901    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465904    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465908    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465910    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465913    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465916    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465918    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465921    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465923    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465926    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465930    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465933    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465935    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465938    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465940    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465943    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465946    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465948    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465951    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465954    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.469392 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465956    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465959    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465961    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465964    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465967    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465969    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465971    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465974    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465976    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465979    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465981    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465984    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465986    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465989    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465992    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.465996    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466000    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466002    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466005    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.469876 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466008    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466011    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466013    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466016    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466019    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466030    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466035    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466038    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466041    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466044    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466046    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466049    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466051    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466054    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466056    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466059    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466061    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466064    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466066    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.470883 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466069    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466072    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466075    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466077    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466080    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466083    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466086    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466088    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466092    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466094    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466097    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466100    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466103    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466105    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466108    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466111    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466113    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466116    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466119    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.471344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466121    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466124    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466126    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466129    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466132    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466134    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466137    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466139    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466142    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466144    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466147    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466150    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466153    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466155    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466158    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466161    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466163    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466166    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466168    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466170    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.471826 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466173    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466175    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466178    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466181    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466185    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466187    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466279    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466287    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466293    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466298    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466303    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466306    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466310    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466315    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466318    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466321    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466324    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466327    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466330    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466333    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466336    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466339    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466342    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 00:55:25.472295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466345    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466348    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466351    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466354    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466357    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466360    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466364    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466367    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466370    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466373    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466377    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466380    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466383    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466386    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466389    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466393    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466396    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466399    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466402    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466405    2574 flags.go:64] FLAG: --enable-server="true"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466408    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466413    2574 flags.go:64] FLAG: --event-burst="100"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466416    2574 flags.go:64] FLAG: --event-qps="50"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466419    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 00:55:25.472941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466422    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466426    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466431    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466434    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466437    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466440    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466443    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466446    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466449    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466452    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466455    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466457    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466460    2574 flags.go:64] FLAG: --feature-gates=""
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466465    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466468    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466471    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466474    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466477    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466481    2574 flags.go:64] FLAG: --help="false"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466484    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466487    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466491    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466494    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 00:55:25.473523 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466497    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466504    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466507    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466510    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466513    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466516    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466518    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466521    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466524    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466527    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466530    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466533    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466536    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466539    2574 flags.go:64] FLAG: --lock-file=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466542    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466545    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466548    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466553    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466556    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466559    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466562    2574 flags.go:64] FLAG: --logging-format="text"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466565    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466569    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466571    2574 flags.go:64] FLAG: --manifest-url=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466574    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 00:55:25.474096 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466579    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466582    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466585    2574 flags.go:64] FLAG: --max-pods="110"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466588    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466591    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466594    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466597    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466601    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466605    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466608    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466615    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466619    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466622    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466625    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466628    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466631    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466634    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466637    2574 flags.go:64] FLAG: --port="10250"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466640    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466643    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-04527091bae342e72"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466646    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466649    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466652    2574 flags.go:64] FLAG: --register-node="true"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466655    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466659    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466662    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 31 00:55:25.474913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466665    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466667    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466671    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466675    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466678    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466681    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466684    2574 flags.go:64] FLAG: --runonce="false"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466686    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466689    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466692    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466695    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466698    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466701    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466705    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466723    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466726    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466729    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466734    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466737    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466740    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466743    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466745    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466751    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466754    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466756    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 00:55:25.475742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466760    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466763    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466766    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466769    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466772    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466775    2574 flags.go:64] FLAG: --v="2"
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466779    2574 flags.go:64] FLAG: --version="false"
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466783    2574 flags.go:64] FLAG: --vmodule=""
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466787    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.466791    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466918    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466921    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466925    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466928    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466930    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466934    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466939    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466942    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466945    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466947    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466950    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.476382 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466955    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466957    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466960    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466963    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466966    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466969    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466972    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466975    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466977    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466980    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466983    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466985    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466988    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466990    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466993    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466996    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.466998    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467001    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467004    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.476926 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467006    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467009    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467011    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467014    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467016    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467019    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467022    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467024    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467027    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467030    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467032    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467035    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467037    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467041    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467044    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467047    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467051    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467054    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467057    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.477372 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467060    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467064    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467067    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467071    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467074    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467076    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467079    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467081    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467084    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467087    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467089    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467092    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467094    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467097    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467100    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467102    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467105    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467107    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467110    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467112    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.477839 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467115    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467117    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467120    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467122    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467125    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467127    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467131    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467134    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467137    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467142    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467144    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467147    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467150    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467152    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467155    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467157    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467160    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467163    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467166    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467168    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.478322 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467171    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467173    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467176    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467178    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467181    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467184    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467187    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467189    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467192    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467195    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467197    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467200    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467203    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467205    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467207    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467210    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.478805 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467213    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.467238    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467273    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467279    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467282    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467287    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467290    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467294    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467298    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467301    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467304    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467307    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467310    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467313    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467316    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.479195 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467319    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467321    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467324    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467326    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467329    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467332    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467334    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467337    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467340    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467342    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467345    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467347    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467350    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467353    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467355    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467358    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467360    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467363    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467365    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.479559 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467368    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467372    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467374    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467378    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467381    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467384    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467387    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467390    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467392    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467394    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467397    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467400    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467402    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467405    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467408    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467411    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467413    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467416    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467420    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.480019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467424    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467428    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467431    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467434    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467436    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467439    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467441    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467444    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467446    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467449    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467452    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467454    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467457    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467460    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467463    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467466    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467470    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467472    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467475    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467478    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.480513 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467481    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467483    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467485    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467488    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467491    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467494    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467497    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467500    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467502    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467505    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467507    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467509    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467512    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467515    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467517    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467520    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467522    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467525    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467528    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467530    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.481019 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467533    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467535    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467538    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467540    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467543    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467545    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467549    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467552    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467555    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467558    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467561    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467565    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467567    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467570    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.467572    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.476164    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.476181    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.476901    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.476908    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476964    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.481596 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476970    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476973    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476977    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476980    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476983    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476986    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476989    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476992    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476994    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476997    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.476999    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477002    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477005    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477008    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477011    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477013    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477016    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477018    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477022    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477024    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.482151 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477027    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477030    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477038    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477041    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477044    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477046    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477049    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477052    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477054    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477057    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477059    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477062    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477064    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477067    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477069    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477072    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477075    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477077    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477079    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.482625 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477082    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477086    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477090    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477093    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477096    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477099    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477102    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477105    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477107    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477110    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477113    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477116    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477118    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477121    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477125    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477129    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477132    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477135    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.483178 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477138    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477140    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477143    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477146    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477148    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477151    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477153    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477156    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477158    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477161    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477163    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477166    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477168    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477171    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477173    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477176    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477178    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477181    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477183    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.483649 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477186    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477188    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477192    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477194    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477197    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477199    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477202    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477204    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477207    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477209    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477212    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477215    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477217    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477220    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477222    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477225    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477227    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477230    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477232    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477235    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.484458 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477237    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477239    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477242    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477245    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477247    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477250    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477252    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477255    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477257    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.477284    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477441    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477445    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477448    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477451    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477454    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477457    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.485185 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477460    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477462    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477465    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477468    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477470    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477473    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477475    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477478    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477480    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477483    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477485    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477488    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477490    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477493    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477495    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477498    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477501    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477503    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477506    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477508    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.485654 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477511    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477513    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477516    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477518    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477521    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477525    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477529    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477532    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477535    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477538    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477541    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477543    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477546    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477549    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477552    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477554    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477557    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477559    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477562    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.486265 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477564    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477567    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477570    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477572    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477575    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477578    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477580    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477583    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477585    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477588    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477590    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477593    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477595    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477598    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477600    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477603    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477605    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477608    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477610    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477612    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.486728 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477615    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477618    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477620    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477623    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477625    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477627    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477630    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477633    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477636    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477638    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477642    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477645    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477648    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477651    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477653    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477656    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477658    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477661    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477663    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.487213 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477666    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477668    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477671    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477673    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477675    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477678    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477680    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477683    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477685    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477687    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477690    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477693    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477695    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477698    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477700    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477730    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477734    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477738    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477740    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477743    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.487734 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477746    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477751    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.477774    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477806    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477810    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477813    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477815    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477818    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477821    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477823    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477826    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477828    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477831    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477834    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477836    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477839    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:55:25.488293 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477842    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477846    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477850    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477853    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477856    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477859    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477862    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477864    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477867    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477870    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477873    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477876    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477879    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477881    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477884    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477886    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477888    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477893    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477896    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:55:25.488793 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477898    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477900    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477903    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477905    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477908    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477910    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477913    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477916    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477918    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477921    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477923    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477928    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477930    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477933    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477936    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477938    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477941    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477943    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477946    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477949    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:55:25.489259 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477951    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477953    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477956    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477959    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477961    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477964    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477966    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477969    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477971    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477974    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477978    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477982    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477984    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477987    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477989    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477992    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477994    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477997    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.477999    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478002    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:55:25.489827 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478004    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478007    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478010    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478015    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478018    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478022    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478032    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478035    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478038    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478041    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478043    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478046    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478049    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478052    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478054    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478057    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478059    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478062    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478065    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:55:25.490315 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478067    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478070    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478072    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478077    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478079    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478082    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478085    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478088    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478091    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478094    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478096    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478099    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478101    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478104    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:25.478106    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.478162    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.480167    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.481187    2574 server.go:1014] "Starting client certificate rotation"
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.481289    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:55:25.490783 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.481336    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:55:25.491231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.483180    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:55:25.491231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.485120    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:55:25.495769 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.495754    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 31 00:55:25.501300 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.501285    2574 log.go:25] "Validated CRI v1 image API"
Oct 31 00:55:25.502088 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.502063    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 00:55:25.505670 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.505652    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 94d64132-da43-4b26-95b1-356216f39f48:/dev/nvme0n1p3 9edc8f53-19b0-485c-b5b3-57909821b9c3:/dev/nvme0n1p4]
Oct 31 00:55:25.505740 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.505670    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 00:55:25.512178 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.512016    2574 manager.go:217] Machine: {Timestamp:2026-10-31 00:55:25.510824052 +0000 UTC m=+0.311362313 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100413 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b06d667be95542c9404c2a5fd1d05 SystemUUID:ec2b06d6-67be-9554-2c94-04c2a5fd1d05 BootID:9d078ce9-753b-4550-a2c1-36c362e2393d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:eb:21:84:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:eb:21:84:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:df:b7:e5:4d:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 00:55:25.512178 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.512177    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 00:55:25.512311 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.512242    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 00:55:25.512311 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.512303    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 00:55:25.515976 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.515940    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 00:55:25.516190 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.515978    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-244.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 00:55:25.516277 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.516196    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 00:55:25.516277 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.516209    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 00:55:25.516277 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.516246    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:55:25.516277 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.516265    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:55:25.516938 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.516925    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 00:55:25.521062 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521047    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 00:55:25.521196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521185    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 00:55:25.521251 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521231    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 00:55:25.521251 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521246    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 00:55:25.521336 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521260    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 00:55:25.521336 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521274    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 31 00:55:25.521336 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521286    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 00:55:25.521632 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.521619    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:55:25.525049 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.525032    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 00:55:25.528686 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.528669    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjghc"
Oct 31 00:55:25.529277 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.529264    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:55:25.530082 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530069    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 00:55:25.530120 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530101    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 00:55:25.530283 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530270    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 00:55:25.530318 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530289    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 00:55:25.530318 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530295    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 00:55:25.530318 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530301    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 00:55:25.530318 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530307    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 00:55:25.530318 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530312    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 00:55:25.530810 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530801    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 00:55:25.530810 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530810    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 00:55:25.530868 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530817    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 00:55:25.530868 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530825    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 00:55:25.530868 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530834    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 00:55:25.530868 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.530842    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 00:55:25.531423 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.531411    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 00:55:25.531470 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.531425    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 00:55:25.534764 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.534748    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-244.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 00:55:25.535444 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.535431    2574 server.go:1263] "Started kubelet"
Oct 31 00:55:25.535550 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.535513    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 00:55:25.535637 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.535538    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 00:55:25.535637 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.535619    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 00:55:25.536157 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.536140    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wjghc"
Oct 31 00:55:25.536364 ip-10-0-140-244 systemd[1]: Started Kubernetes Kubelet.
Oct 31 00:55:25.536993 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.536704    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 00:55:25.537746 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.537700    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 00:55:25.540160 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.540110    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 00:55:25.541558 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.541539    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-244.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 00:55:25.541618 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.541562    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 00:55:25.542597 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.541652    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-244.us-east-2.compute.internal.18e3773033d6391e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-244.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-244.us-east-2.compute.internal,},FirstTimestamp:2026-10-31 00:55:25.53540227 +0000 UTC m=+0.335940532,LastTimestamp:2026-10-31 00:55:25.53540227 +0000 UTC m=+0.335940532,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-244.us-east-2.compute.internal,}"
Oct 31 00:55:25.543020 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.542996    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 00:55:25.543293 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.543271    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 00:55:25.544186 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.544165    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:25.544342 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544321    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 00:55:25.544427 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544345    2574 factory.go:55] Registering systemd factory
Oct 31 00:55:25.544427 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544360    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 31 00:55:25.544671 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544656    2574 factory.go:153] Registering CRI-O factory
Oct 31 00:55:25.544764 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544677    2574 factory.go:223] Registration of the crio container factory successfully
Oct 31 00:55:25.544816 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544774    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 00:55:25.544816 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544794    2574 factory.go:103] Registering Raw factory
Oct 31 00:55:25.544816 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.544805    2574 manager.go:1199] Started watching for new ooms in manager
Oct 31 00:55:25.545778 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.545765    2574 manager.go:319] Starting recovery of all containers
Oct 31 00:55:25.547388 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.547185    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 00:55:25.547388 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.547389    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 00:55:25.547537 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.547530    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 00:55:25.547586 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.547539    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 00:55:25.548151 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.548133    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:25.550319 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.550292    2574 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-244.us-east-2.compute.internal\" not found" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.558927 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.558911    2574 manager.go:324] Recovery completed
Oct 31 00:55:25.563317 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.563305    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.565524 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.565511    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.565579 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.565537    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.565579 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.565548    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.566110 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566097    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 31 00:55:25.566110 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566108    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 00:55:25.566219 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566123    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 00:55:25.566219 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566174    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.566921 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566906    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.566978 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566933    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.566978 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.566943    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.568269 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.568257    2574 policy_none.go:50] "Start"
Oct 31 00:55:25.568305 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.568273    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 00:55:25.568305 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.568283    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 00:55:25.570297 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.570285    2574 policy_none.go:44] "Start"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.623045    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.623350    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.624503    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.624526    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.624547    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.624581    2574 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628038    2574 manager.go:341] "Starting Device Plugin manager"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628167    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.628504    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628516    2574 server.go:87] "Starting device plugin registration server"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628765    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628777    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628872    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628947    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.628954    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.629340    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 00:55:25.632077 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.629383    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:25.725730 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.725657    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"]
Oct 31 00:55:25.725730 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.725726    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.729064 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.729050    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.730819 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730797    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.730909 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730832    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.730909 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730849    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.730909 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730797    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.731034 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730912    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.731034 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730926    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.731034 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.730961    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.733079 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.733068    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.733238 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.733224    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.733286 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.733252    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.736307 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.736287    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.736410 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.736313    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.736410 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.736323    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.736410 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.736293    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.736410 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.736405    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.736555 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.736421    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.737233 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.737220    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.737272 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.737240    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-244.us-east-2.compute.internal\": node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:25.738495 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.738478    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.738543 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.738515    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:55:25.739258 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.739243    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:55:25.739335 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.739275    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:55:25.739335 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.739289    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:55:25.748947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.748929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.749031 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.748959    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.749031 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.748989    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/751e5bfb25820222e24aac2c7122de2e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"751e5bfb25820222e24aac2c7122de2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.752980 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.752964    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:25.759188 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.759166    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.761563 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.761546    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.849176 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.849143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.849289 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.849179    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.849289 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.849202    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/751e5bfb25820222e24aac2c7122de2e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"751e5bfb25820222e24aac2c7122de2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.849289 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.849231    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.849289 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.849240    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.849289 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:25.849269    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/751e5bfb25820222e24aac2c7122de2e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"751e5bfb25820222e24aac2c7122de2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:25.853173 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.853157    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:25.953787 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:25.953756    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.054530 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.054474    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.062641 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.062621    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:26.064770 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.064756    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:26.154708 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.154686    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.255156 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.255133    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.355625 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.355584    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.407607 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.407178    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:26.456041 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.456014    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.481166 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.481144    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 00:55:26.481602 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.481269    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 00:55:26.481602 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.481322    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 00:55:26.481602 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.481324    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 00:55:26.542793 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.542738    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 00:50:26 +0000 UTC" deadline="2028-08-17 20:44:29.890447142 +0000 UTC"
Oct 31 00:55:26.542793 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.542786    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15763h49m3.347663514s"
Oct 31 00:55:26.556289 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.556263    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.656490 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.656436    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.756966 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.756935    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.780093 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:26.780058    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d2bb5ec637c932bd8834e531979055.slice/crio-92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b WatchSource:0}: Error finding container 92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b: Status 404 returned error can't find the container with id 92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b
Oct 31 00:55:26.784683 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:26.784657    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751e5bfb25820222e24aac2c7122de2e.slice/crio-f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23 WatchSource:0}: Error finding container f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23: Status 404 returned error can't find the container with id f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23
Oct 31 00:55:26.786595 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:26.786581    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:55:26.857701 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.857674    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:26.958156 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:26.958135    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-244.us-east-2.compute.internal\" not found"
Oct 31 00:55:27.046017 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:27.046000    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:27.145199 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:27.145177    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:27.155537 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:27.155516    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:55:27.156651 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:27.156639    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:55:27.165920 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:27.165905    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:55:27.413366 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:27.413291    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:28.920148 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.920125    2574 apiserver.go:52] "Watching apiserver"
Oct 31 00:55:28.928682 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.928659    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:28.928933 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.928916    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 00:55:28.929281 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.929260    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-d6xbt","openshift-multus/network-metrics-daemon-7r985","openshift-network-diagnostics/network-check-target-rs9gd","kube-system/global-pull-secret-syncer-6tsqv","openshift-image-registry/node-ca-hjrz2","openshift-dns/node-resolver-ddx8v","openshift-multus/multus-2m96p","openshift-network-operator/iptables-alerter-vh94r","kube-system/konnectivity-agent-m9m29","openshift-cluster-node-tuning-operator/tuned-tmhd2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-2z92c","kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"]
Oct 31 00:55:28.932011 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.931991    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.934341 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.934321    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:28.936317 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.936293    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 00:55:28.936425 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.936409    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 00:55:28.936522 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.936507    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 00:55:28.936638 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.936622    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xxmpd\""
Oct 31 00:55:28.936714 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.936670    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 00:55:28.939494 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.939472    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 00:55:28.939585 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.939560    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 00:55:28.939664 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.939632    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:55:28.939733 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.939714    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-89rf9\""
Oct 31 00:55:28.939784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.939762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:28.939856 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:28.939830    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:28.940025 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.940011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:28.940086 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:28.940062    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:28.943054 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.941314    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:28.943054 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.941730    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 00:55:28.943995 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.943976    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:28.947011 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.946622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:28.950177 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.950159    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:28.952538 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.952520    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:28.952681 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:28.952590    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:28.956617 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-multus\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956705 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-kubelet\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956705 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-system-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956811 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-etc-kubernetes\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956811 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-host\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:28.956811 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956770    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-multus-daemon-config\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956948 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-multus-certs\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956948 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:28.956948 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956948 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-cnibin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.956948 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956925    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-k8s-cni-cncf-io\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-netns\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.956976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-conf-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea64b244-e85f-494e-a824-9eba8ccfe22c-tmp-dir\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-hostroot\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb321839-52ad-47ca-b4d6-b30b18970649-iptables-alerter-script\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2j2l\" (UniqueName: \"kubernetes.io/projected/fb321839-52ad-47ca-b4d6-b30b18970649-kube-api-access-g2j2l\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957115    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:28.957164 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-kubelet-config\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957171    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-dbus\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-serviceca\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-socket-dir-parent\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7kl8\" (UniqueName: \"kubernetes.io/projected/8b4e40a1-88e5-493b-901d-668e910172cd-kube-api-access-j7kl8\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb321839-52ad-47ca-b4d6-b30b18970649-host-slash\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957315    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea64b244-e85f-494e-a824-9eba8ccfe22c-hosts-file\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957342    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zthpv\" (UniqueName: \"kubernetes.io/projected/ea64b244-e85f-494e-a824-9eba8ccfe22c-kube-api-access-zthpv\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-cni-binary-copy\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-bin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:28.957623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.957423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ftlv\" (UniqueName: \"kubernetes.io/projected/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-kube-api-access-4ftlv\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:28.959333 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.959301    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 00:55:28.960337 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.960316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:28.963010 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.962990    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:28.965720 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.965701    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:28.977943 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.977925    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2l2t"
Oct 31 00:55:28.984058 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:28.984040    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n2l2t"
Oct 31 00:55:29.027214 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.027175    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"f6d2bb5ec637c932bd8834e531979055","Type":"ContainerStarted","Data":"92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b"}
Oct 31 00:55:29.028890 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.028866    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"751e5bfb25820222e24aac2c7122de2e","Type":"ContainerStarted","Data":"f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23"}
Oct 31 00:55:29.036728 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.036708    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 00:55:29.036812 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.036800    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:55:29.036959 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.036937    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jrg45\""
Oct 31 00:55:29.037005 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.036995    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 00:55:29.037057 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.037048    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 00:55:29.037088 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.037053    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 00:55:29.037088 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.036944    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 00:55:29.037286 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.037257    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 00:55:29.038337 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.038315    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 00:55:29.038462 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.038431    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 00:55:29.038462 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.038440    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 00:55:29.038566 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.038477    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 00:55:29.038566 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.038525    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 00:55:29.038566 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.038535    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m2wnh\""
Oct 31 00:55:29.039879 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.039862    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040082    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4j5v2\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040102    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040133    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040157    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040164    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040188    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xcm4p\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040192    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040212    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040089    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j9qsc\""
Oct 31 00:55:29.040402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040257    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dlpqk\""
Oct 31 00:55:29.040908 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040412    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9dpj5\""
Oct 31 00:55:29.040908 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.040481    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 00:55:29.042977 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.042959    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 00:55:29.058322 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-system-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058324    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-etc-kubernetes\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058351    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-var-lib-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.058402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058389    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-bin\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-env-overrides\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058429    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-system-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058438    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058479    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-var-lib-kubelet\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058530    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-run\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.058538    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:29.058565 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058553    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-host\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-cnibin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.058619    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:29.558575562 +0000 UTC m=+2.961226697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-k8s-cni-cncf-io\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058652    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-cnibin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-netns\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058709    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea64b244-e85f-494e-a824-9eba8ccfe22c-tmp-dir\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058772    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-netns\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058804    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-etc-kubernetes\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v884q\" (UniqueName: \"kubernetes.io/projected/41c16bf9-670b-4e0d-b009-8898ccad69b5-kube-api-access-v884q\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058866    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-hostroot\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058884    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-k8s-cni-cncf-io\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058889    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2j2l\" (UniqueName: \"kubernetes.io/projected/fb321839-52ad-47ca-b4d6-b30b18970649-kube-api-access-g2j2l\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-serviceca\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.058969 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.058970    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea64b244-e85f-494e-a824-9eba8ccfe22c-hosts-file\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.059662 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-socket-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.059662 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.059662 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb321839-52ad-47ca-b4d6-b30b18970649-host-slash\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.059662 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059116    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2ptv\" (UniqueName: \"kubernetes.io/projected/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-kube-api-access-k2ptv\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.059662 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-tuned\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/852b2fe1-0192-404e-9c39-e16b622a3b9f-agent-certs\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-tmp\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059737    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-kubelet\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059763    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-systemd-units\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059785    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-systemd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059813    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-modprobe-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-conf\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059862    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-etc-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.059903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059892    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90525854-b0fe-4589-8f36-df21ad285d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-host\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059959    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-kubernetes\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.059988    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-systemd\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060012    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-device-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060099    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8hhn\" (UniqueName: \"kubernetes.io/projected/3005fee5-7711-481a-89dc-8f191ef548cd-kube-api-access-m8hhn\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-multus-daemon-config\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-multus-certs\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060180    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysconfig\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-registration-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-ovn\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060273    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-netd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.060328 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-conf-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060382    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cnibin\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060407    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-sys-fs\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060489    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-script-lib\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060516    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb321839-52ad-47ca-b4d6-b30b18970649-iptables-alerter-script\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-kubelet-config\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-dbus\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-slash\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060676    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-node-log\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060700    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-socket-dir-parent\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7kl8\" (UniqueName: \"kubernetes.io/projected/8b4e40a1-88e5-493b-901d-668e910172cd-kube-api-access-j7kl8\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.060954 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zthpv\" (UniqueName: \"kubernetes.io/projected/ea64b244-e85f-494e-a824-9eba8ccfe22c-kube-api-access-zthpv\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060776    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/852b2fe1-0192-404e-9c39-e16b622a3b9f-konnectivity-ca\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060962    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-netns\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.060987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-log-socket\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061011    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-lib-modules\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w92g4\" (UniqueName: \"kubernetes.io/projected/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-kube-api-access-w92g4\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061083    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf85m\" (UniqueName: \"kubernetes.io/projected/90525854-b0fe-4589-8f36-df21ad285d0b-kube-api-access-tf85m\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061110    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-cni-binary-copy\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-system-cni-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061190    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-bin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ftlv\" (UniqueName: \"kubernetes.io/projected/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-kube-api-access-4ftlv\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-sys\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.061715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-config\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-multus\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061367    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-kubelet\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061435    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-kubelet\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061725    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea64b244-e85f-494e-a824-9eba8ccfe22c-tmp-dir\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.061991    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-hostroot\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062242    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea64b244-e85f-494e-a824-9eba8ccfe22c-hosts-file\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062258    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb321839-52ad-47ca-b4d6-b30b18970649-host-slash\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062267    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-host\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062308    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-multus-certs\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062347    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-conf-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062455    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-socket-dir-parent\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062455    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-dbus\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:29.062903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062465    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062712    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-multus\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.062903 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.062750    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-kubelet-config\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:29.064149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.063452    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb321839-52ad-47ca-b4d6-b30b18970649-iptables-alerter-script\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.064149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.063828    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-cni-binary-copy\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.064149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.063889    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-bin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.064149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.064062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-serviceca\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.064149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.064076    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-multus-daemon-config\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.068288 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.068269    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:55:29.068369 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.068292    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:55:29.068369 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.068305    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:29.068369 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.068354    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:29.568338649 +0000 UTC m=+2.970989772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:29.068533 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.068273    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 00:55:29.072447 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.072422    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2j2l\" (UniqueName: \"kubernetes.io/projected/fb321839-52ad-47ca-b4d6-b30b18970649-kube-api-access-g2j2l\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.072529 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.072433    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7kl8\" (UniqueName: \"kubernetes.io/projected/8b4e40a1-88e5-493b-901d-668e910172cd-kube-api-access-j7kl8\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.072961 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.072934    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zthpv\" (UniqueName: \"kubernetes.io/projected/ea64b244-e85f-494e-a824-9eba8ccfe22c-kube-api-access-zthpv\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.073904 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.073882    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ftlv\" (UniqueName: \"kubernetes.io/projected/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-kube-api-access-4ftlv\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.162612 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cnibin\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.162735 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.162735 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162652    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.162735 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162677    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cnibin\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.162735 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-sys-fs\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.162735 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162736    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-script-lib\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-sys-fs\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162792    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-slash\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-node-log\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162832    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/852b2fe1-0192-404e-9c39-e16b622a3b9f-konnectivity-ca\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162866    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-netns\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-log-socket\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-lib-modules\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.162952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162937    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w92g4\" (UniqueName: \"kubernetes.io/projected/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-kube-api-access-w92g4\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf85m\" (UniqueName: \"kubernetes.io/projected/90525854-b0fe-4589-8f36-df21ad285d0b-kube-api-access-tf85m\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-system-cni-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.162993    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-sys\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-config\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163102    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-var-lib-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-bin\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163138    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-env-overrides\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163184    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-var-lib-kubelet\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-run\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-host\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v884q\" (UniqueName: \"kubernetes.io/projected/41c16bf9-670b-4e0d-b009-8898ccad69b5-kube-api-access-v884q\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-socket-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.163282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2ptv\" (UniqueName: \"kubernetes.io/projected/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-kube-api-access-k2ptv\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163337    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-tuned\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/852b2fe1-0192-404e-9c39-e16b622a3b9f-agent-certs\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163387    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-tmp\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-kubelet\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-systemd-units\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-systemd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-modprobe-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-conf\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-etc-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90525854-b0fe-4589-8f36-df21ad285d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-kubernetes\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-systemd\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-device-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8hhn\" (UniqueName: \"kubernetes.io/projected/3005fee5-7711-481a-89dc-8f191ef548cd-kube-api-access-m8hhn\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163627    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysconfig\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.163844 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163642    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163657    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-registration-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-ovn\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163691    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-netd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163751    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-netd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-var-lib-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163821    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-bin\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164047    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-config\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164122    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-env-overrides\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164228    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164261    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-var-lib-kubelet\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164289    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-run\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.164362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164318    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-host\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.164728 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164498    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-script-lib\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164728 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164555    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-socket-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.164728 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.164580    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:29.164728 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164596    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164728 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.164654    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:29.664638405 +0000 UTC m=+3.067289521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:29.164728 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164696    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-slash\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.164915 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.164734    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-node-log\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165160 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165136    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.165251 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165196    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-system-cni-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.165414 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165331    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165527 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165437    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-netns\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165527 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165469    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-log-socket\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165632 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165536    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-etc-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165632 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163335    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.165743 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165710    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165792 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165764    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-sys\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.165842 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165816    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-systemd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.165842 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165830    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-kubernetes\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.165934 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165882    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-device-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.165934 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.165925    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-kubelet\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.166056 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166031    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-systemd-units\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.166145 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166088    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-systemd\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.166145 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.163440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.166145 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166138    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-modprobe-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.166291 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166188    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.166291 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166205    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-conf\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.166291 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysconfig\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.166291 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166285    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-registration-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.166473 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166328    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-ovn\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.166473 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.166425    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-lib-modules\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.167426 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.167396    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-tuned\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.167575 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.167560    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.167692 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.167670    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.167692 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.167683    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/852b2fe1-0192-404e-9c39-e16b622a3b9f-konnectivity-ca\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.170019 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.169963    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/852b2fe1-0192-404e-9c39-e16b622a3b9f-agent-certs\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.170282 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.170260    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90525854-b0fe-4589-8f36-df21ad285d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.170519 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.170501    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-tmp\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.176241 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.176218    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v884q\" (UniqueName: \"kubernetes.io/projected/41c16bf9-670b-4e0d-b009-8898ccad69b5-kube-api-access-v884q\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:29.176868 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.176844    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf85m\" (UniqueName: \"kubernetes.io/projected/90525854-b0fe-4589-8f36-df21ad285d0b-kube-api-access-tf85m\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.176964 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.176945    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8hhn\" (UniqueName: \"kubernetes.io/projected/3005fee5-7711-481a-89dc-8f191ef548cd-kube-api-access-m8hhn\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.178438 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.178299    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2ptv\" (UniqueName: \"kubernetes.io/projected/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-kube-api-access-k2ptv\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.178920 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.178901    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w92g4\" (UniqueName: \"kubernetes.io/projected/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-kube-api-access-w92g4\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.251938 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.251904    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2m96p"
Oct 31 00:55:29.261665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.261613    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:55:29.262688 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.262664    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4e40a1_88e5_493b_901d_668e910172cd.slice/crio-cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c WatchSource:0}: Error finding container cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c: Status 404 returned error can't find the container with id cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c
Oct 31 00:55:29.271863 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.271839    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb321839_52ad_47ca_b4d6_b30b18970649.slice/crio-7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a WatchSource:0}: Error finding container 7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a: Status 404 returned error can't find the container with id 7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a
Oct 31 00:55:29.274405 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.274319    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:55:29.277916 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.277895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:55:29.284624 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.284581    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a21f30_84ee_4b17_93f6_8da1e8ecdf0a.slice/crio-f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a WatchSource:0}: Error finding container f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a: Status 404 returned error can't find the container with id f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a
Oct 31 00:55:29.288756 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.288734    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:29.298288 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.298264    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852b2fe1_0192_404e_9c39_e16b622a3b9f.slice/crio-e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb WatchSource:0}: Error finding container e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb: Status 404 returned error can't find the container with id e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb
Oct 31 00:55:29.310055 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.310033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:55:29.312446 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.312390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:55:29.319977 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.319958    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:55:29.321430 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.321403    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955e9bb3_d8e7_419d_9d8f_7c9af8d88e07.slice/crio-f3ed9364546df8a6a82bacde95ca75ca4d1ce92218ace050be6ea649f67e48e3 WatchSource:0}: Error finding container f3ed9364546df8a6a82bacde95ca75ca4d1ce92218ace050be6ea649f67e48e3: Status 404 returned error can't find the container with id f3ed9364546df8a6a82bacde95ca75ca4d1ce92218ace050be6ea649f67e48e3
Oct 31 00:55:29.330103 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.330081    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0eff54_56e9_4ff4_9c31_17ca3325e0a6.slice/crio-83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1 WatchSource:0}: Error finding container 83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1: Status 404 returned error can't find the container with id 83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1
Oct 31 00:55:29.330915 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.330652    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3005fee5_7711_481a_89dc_8f191ef548cd.slice/crio-7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41 WatchSource:0}: Error finding container 7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41: Status 404 returned error can't find the container with id 7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41
Oct 31 00:55:29.338111 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.338092    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:29.348964 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:55:29.348940    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90525854_b0fe_4589_8f36_df21ad285d0b.slice/crio-e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e WatchSource:0}: Error finding container e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e: Status 404 returned error can't find the container with id e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e
Oct 31 00:55:29.566295 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.566214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:29.566453 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.566330    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:29.566453 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.566395    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:30.566376569 +0000 UTC m=+3.969027690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.667291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.667345    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.667485    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.667487    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.667521    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.667535    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.667548    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:30.667530472 +0000 UTC m=+4.070181595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:29.667890 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:29.667591    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:30.667573473 +0000 UTC m=+4.070224593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:29.988562 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.988258    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:50:28 +0000 UTC" deadline="2028-03-27 17:48:05.924690454 +0000 UTC"
Oct 31 00:55:29.988562 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:29.988310    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12328h52m35.936383276s"
Oct 31 00:55:30.041330 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.041200    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk" event={"ID":"3005fee5-7711-481a-89dc-8f191ef548cd","Type":"ContainerStarted","Data":"7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41"}
Oct 31 00:55:30.054005 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.053965    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m9m29" event={"ID":"852b2fe1-0192-404e-9c39-e16b622a3b9f","Type":"ContainerStarted","Data":"e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb"}
Oct 31 00:55:30.061714 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.061686    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ddx8v" event={"ID":"ea64b244-e85f-494e-a824-9eba8ccfe22c","Type":"ContainerStarted","Data":"85380a130db8fe5600daeda9f2fc60cfeb53199f87289a9ab6bef2d1bc18a471"}
Oct 31 00:55:30.065502 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.065174    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hjrz2" event={"ID":"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a","Type":"ContainerStarted","Data":"f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a"}
Oct 31 00:55:30.070168 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.070097    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vh94r" event={"ID":"fb321839-52ad-47ca-b4d6-b30b18970649","Type":"ContainerStarted","Data":"7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a"}
Oct 31 00:55:30.075207 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.075111    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2m96p" event={"ID":"8b4e40a1-88e5-493b-901d-668e910172cd","Type":"ContainerStarted","Data":"cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c"}
Oct 31 00:55:30.084569 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.084491    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e"}
Oct 31 00:55:30.090825 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.090796    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerStarted","Data":"83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1"}
Oct 31 00:55:30.095118 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.094769    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tmhd2" event={"ID":"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07","Type":"ContainerStarted","Data":"f3ed9364546df8a6a82bacde95ca75ca4d1ce92218ace050be6ea649f67e48e3"}
Oct 31 00:55:30.388841 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.388804    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:55:30.574959 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.574910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:30.575087 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.575033    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:30.575146 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.575092    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:32.575074533 +0000 UTC m=+5.977725655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.675805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.675853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.675985    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.676047    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:32.676029676 +0000 UTC m=+6.078680802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.676425    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.676442    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.676453    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:30.676572 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:30.676494    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:32.67648139 +0000 UTC m=+6.079132499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:30.988756 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.988668    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:50:28 +0000 UTC" deadline="2028-04-20 01:53:42.642478379 +0000 UTC"
Oct 31 00:55:30.988756 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:30.988700    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12888h58m11.653782121s"
Oct 31 00:55:31.025409 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.025374    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:31.025576 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:31.025522    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:31.025936 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.025914    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:31.026040 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:31.026008    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:31.026098 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.026064    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:31.026149 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:31.026134    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:31.104696 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.104660    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"751e5bfb25820222e24aac2c7122de2e","Type":"ContainerStarted","Data":"1ef24b55310bedc8525b6bdc27b260f7cb043d17b91f69a65df4d530e339d664"}
Oct 31 00:55:31.114493 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.114346    2574 generic.go:356] "Generic (PLEG): container finished" podID="f6d2bb5ec637c932bd8834e531979055" containerID="a322d04abc7a727b5b1ca408105b58dd9e576ab819a85fbec971cea11ea85cb7" exitCode=0
Oct 31 00:55:31.114493 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.114395    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"f6d2bb5ec637c932bd8834e531979055","Type":"ContainerDied","Data":"a322d04abc7a727b5b1ca408105b58dd9e576ab819a85fbec971cea11ea85cb7"}
Oct 31 00:55:31.135787 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:31.134963    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal" podStartSLOduration=3.134946004 podStartE2EDuration="3.134946004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:31.120237829 +0000 UTC m=+4.522888955" watchObservedRunningTime="2026-10-31 00:55:31.134946004 +0000 UTC m=+4.537597132"
Oct 31 00:55:32.128535 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:32.128495    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"f6d2bb5ec637c932bd8834e531979055","Type":"ContainerStarted","Data":"e69eebfaa35701d8f33f2272753b64f183a3d5f61ca713311f04549de4d999ee"}
Oct 31 00:55:32.592948 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:32.592255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:32.592948 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.592483    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:32.592948 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.592548    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:36.592529348 +0000 UTC m=+9.995180452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:32.692760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:32.692816    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.692945    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.693008    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:36.692990473 +0000 UTC m=+10.095641578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.693490    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.693511    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.693526    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:32.693692 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:32.693578    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:36.693560971 +0000 UTC m=+10.096212075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:33.025725 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:33.022932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:33.025725 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:33.023068    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:33.025725 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:33.023440    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:33.025725 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:33.023532    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:33.025725 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:33.023642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:33.025725 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:33.023727    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:33.136855 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:33.133383    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="47c7ca80308dab4ae7b9e2a7e172a7b3180ab2b886da0e49960ab023e6877b18" exitCode=0
Oct 31 00:55:33.136855 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:33.134437    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerDied","Data":"47c7ca80308dab4ae7b9e2a7e172a7b3180ab2b886da0e49960ab023e6877b18"}
Oct 31 00:55:33.160316 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:33.159537    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" podStartSLOduration=5.159520976 podStartE2EDuration="5.159520976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:32.145304675 +0000 UTC m=+5.547955796" watchObservedRunningTime="2026-10-31 00:55:33.159520976 +0000 UTC m=+6.562172102"
Oct 31 00:55:34.147452 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:34.146912    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m9m29" event={"ID":"852b2fe1-0192-404e-9c39-e16b622a3b9f","Type":"ContainerStarted","Data":"568cf7142361c822df58cfd00390bc2fe892ec8fc6736cd0930fb12e83b744a5"}
Oct 31 00:55:34.162399 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:34.162354    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m9m29" podStartSLOduration=3.5103571369999997 podStartE2EDuration="7.162337002s" totalImagesPullingTime="3.651979865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:34.161881329 +0000 UTC m=+7.564532456" watchObservedRunningTime="2026-10-31 00:55:34.162337002 +0000 UTC m=+7.564988129"
Oct 31 00:55:35.024220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:35.023702    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:35.024220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:35.023703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:35.024220 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:35.023849    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:35.024220 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:35.023927    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:35.024220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:35.023977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:35.024220 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:35.024049    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:35.154331 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:35.154270    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk" event={"ID":"3005fee5-7711-481a-89dc-8f191ef548cd","Type":"ContainerStarted","Data":"71f1f3c786430ec57f11085b0793630768d20118ec8d367e649278cafc942c90"}
Oct 31 00:55:35.158822 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:35.158797    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hjrz2" event={"ID":"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a","Type":"ContainerStarted","Data":"11ba80eb48c6dcd034b3192cde3d8c15edc63b5827f16a62e847de19325df170"}
Oct 31 00:55:36.626024 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:36.625991    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:36.626438 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.626178    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:36.626438 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.626229    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:44.626212219 +0000 UTC m=+18.028863336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:36.640154 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:36.640133    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:36.726550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:36.726592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.726769    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.726816    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:44.726802061 +0000 UTC m=+18.129453178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.726886    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.726900    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.726911    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:36.727218 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:36.726946    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:55:44.726935155 +0000 UTC m=+18.129586272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:37.023701 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.023615    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:37.023701 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.023661    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:37.023947 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:37.023735    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:37.023947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.023623    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:37.023947 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:37.023856    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:37.023947 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:37.023944    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:37.035947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.035678    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T00:55:36.640150268Z","UUID":"ffc91abc-8a15-4321-927b-7728e86c469c","Handler":null,"Name":"","Endpoint":""}
Oct 31 00:55:37.037335 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.037306    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 00:55:37.037440 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.037342    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 00:55:37.165316 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.165068    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk" event={"ID":"3005fee5-7711-481a-89dc-8f191ef548cd","Type":"ContainerStarted","Data":"5507413fad9470f9e5ea6cea1d867fdb0e014ed41c9bf5dfc32ffc6cda1588c1"}
Oct 31 00:55:37.167174 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.167143    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2m96p" event={"ID":"8b4e40a1-88e5-493b-901d-668e910172cd","Type":"ContainerStarted","Data":"b72334bae240686314644e30359c514d978b8e91f4e5cbcd226310f8b0fbeef1"}
Oct 31 00:55:37.185280 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.184420    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-2m96p" podStartSLOduration=2.861570545 podStartE2EDuration="10.184404305s" totalImagesPullingTime="7.32283376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:37.184343591 +0000 UTC m=+10.586994716" watchObservedRunningTime="2026-10-31 00:55:37.184404305 +0000 UTC m=+10.587055431"
Oct 31 00:55:37.185280 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.185117    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hjrz2" podStartSLOduration=5.323191304 podStartE2EDuration="10.185107738s" totalImagesPullingTime="4.861916434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:35.173699158 +0000 UTC m=+8.576350285" watchObservedRunningTime="2026-10-31 00:55:37.185107738 +0000 UTC m=+10.587758913"
Oct 31 00:55:37.876048 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.876010    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:37.877446 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:37.876620    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:38.171998 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.171365    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk" event={"ID":"3005fee5-7711-481a-89dc-8f191ef548cd","Type":"ContainerStarted","Data":"91817b6b740329d392f383d0ce69e35bf33cbe8d4693b440f9646dbb7c0e0882"}
Oct 31 00:55:38.173659 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.173626    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ddx8v" event={"ID":"ea64b244-e85f-494e-a824-9eba8ccfe22c","Type":"ContainerStarted","Data":"1fc44951582372466c89d962d224f3e74a9f7f7c8e607c9a5b57cfe2996d1730"}
Oct 31 00:55:38.175754 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.175722    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vh94r" event={"ID":"fb321839-52ad-47ca-b4d6-b30b18970649","Type":"ContainerStarted","Data":"6116be9c577240bc3581f2fdd69b58575c4ad417843faf3227a6845d8d6af053"}
Oct 31 00:55:38.178522 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.178026    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="3416f279a014ea3542d0d8bb56ab32448a57964b72e0048837304155e33fc558" exitCode=0
Oct 31 00:55:38.178522 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.178132    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerDied","Data":"3416f279a014ea3542d0d8bb56ab32448a57964b72e0048837304155e33fc558"}
Oct 31 00:55:38.189009 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.188962    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk" podStartSLOduration=2.978258318 podStartE2EDuration="11.188950488s" totalImagesPullingTime="8.21069217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:38.188520667 +0000 UTC m=+11.591171794" watchObservedRunningTime="2026-10-31 00:55:38.188950488 +0000 UTC m=+11.591601612"
Oct 31 00:55:38.202593 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.202548    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-ddx8v" podStartSLOduration=3.214842483 podStartE2EDuration="11.202532959s" totalImagesPullingTime="7.987690476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:38.202287035 +0000 UTC m=+11.604938162" watchObservedRunningTime="2026-10-31 00:55:38.202532959 +0000 UTC m=+11.605184088"
Oct 31 00:55:38.237359 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.237279    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vh94r" podStartSLOduration=3.877607227 podStartE2EDuration="11.237262035s" totalImagesPullingTime="7.359654808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:38.236627174 +0000 UTC m=+11.639278300" watchObservedRunningTime="2026-10-31 00:55:38.237262035 +0000 UTC m=+11.639913353"
Oct 31 00:55:38.993489 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.993400    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:38.994533 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:38.994343    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:55:39.023820 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:39.023443    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:39.023820 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:39.023540    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:39.023820 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:39.023685    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:39.023820 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:39.023773    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:39.024116 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:39.023911    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:39.024116 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:39.024002    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:40.185193 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:40.185163    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="dbae7b9fb41c4f3aea8756bca412171409ebc074cf2f3e50eb80339469a303c1" exitCode=0
Oct 31 00:55:40.185675 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:40.185238    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerDied","Data":"dbae7b9fb41c4f3aea8756bca412171409ebc074cf2f3e50eb80339469a303c1"}
Oct 31 00:55:41.023436 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:41.023205    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:41.023595 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:41.023227    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:41.023595 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:41.023541    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:41.023595 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:41.023274    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:41.023786 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:41.023659    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:41.023786 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:41.023687    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:41.189157 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:41.189121    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tmhd2" event={"ID":"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07","Type":"ContainerStarted","Data":"a58d3dc2c4ccdc51395a6d94d452e14d739200c19b195ab73a16b3decbb162a7"}
Oct 31 00:55:41.206389 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:41.206339    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tmhd2" podStartSLOduration=3.401310983 podStartE2EDuration="14.206324552s" totalImagesPullingTime="10.805013569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:41.206108767 +0000 UTC m=+14.608759893" watchObservedRunningTime="2026-10-31 00:55:41.206324552 +0000 UTC m=+14.608975678"
Oct 31 00:55:42.196929 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:42.196897    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="d49d286b06c2ecef0b91a6ca50ea985cff4d93c61c66be485e1878c6a6ed3ec2" exitCode=0
Oct 31 00:55:42.197395 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:42.196985    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerDied","Data":"d49d286b06c2ecef0b91a6ca50ea985cff4d93c61c66be485e1878c6a6ed3ec2"}
Oct 31 00:55:43.023057 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:43.023024    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:43.023249 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:43.023100    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:43.023249 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:43.023116    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:43.023249 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:43.023202    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:43.023406 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:43.023308    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:43.023452 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:43.023435    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:44.202917 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.202756    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:55:44.203367 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203238    2574 generic.go:356] "Generic (PLEG): container finished" podID="90525854-b0fe-4589-8f36-df21ad285d0b" containerID="dde362c46632d8a6ba801a42bdcaa8bed38d9c94ef2ec2738d5a40ef63980b28" exitCode=1
Oct 31 00:55:44.203367 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203304    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"06b80b16748967166ae4387ecadc039284540b1f610c82de83137b24bf51f487"}
Oct 31 00:55:44.203447 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203374    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"93b094be00a9208f5088746f49b5b6e3aa6bb5f7f630bad2aa1e419b25149e70"}
Oct 31 00:55:44.203447 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203411    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"347db85d62be55396a8f383a9e0095c035fffde8232722fbc14d50ebd28c286b"}
Oct 31 00:55:44.203447 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203439    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"641428284adad797959c0a168ceff0dd54ffc2b3c977d015d67d2dc45dabb64e"}
Oct 31 00:55:44.203538 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203472    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerDied","Data":"dde362c46632d8a6ba801a42bdcaa8bed38d9c94ef2ec2738d5a40ef63980b28"}
Oct 31 00:55:44.203538 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.203520    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"34d009dd91873aee51cf0b19db283dc0e0e020fc8686ae9e718b02b472cbd257"}
Oct 31 00:55:44.205822 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.205803    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="b088c4dc1cd885acd45ec3a1915691771720f6fe667df64e6fd10d6f7ca5a3c5" exitCode=0
Oct 31 00:55:44.205887 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.205832    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerDied","Data":"b088c4dc1cd885acd45ec3a1915691771720f6fe667df64e6fd10d6f7ca5a3c5"}
Oct 31 00:55:44.681576 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.681549    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:44.681723 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.681683    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:44.681763 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.681741    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:00.681724835 +0000 UTC m=+34.084375955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:55:44.781901 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.781877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:44.782006 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:44.781906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:44.782006 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.781997    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:44.782068 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.782024    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:55:44.782068 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.782040    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:55:44.782068 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.782049    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:00.782033887 +0000 UTC m=+34.184685005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:55:44.782068 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.782053    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:44.782181 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:44.782086    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:00.782076585 +0000 UTC m=+34.184727706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:55:45.023323 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:45.023245    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:45.023323 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:45.023272    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:45.023489 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:45.023358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:45.023489 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:45.023368    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:45.023489 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:45.023418    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:45.023584 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:45.023493    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:45.210805 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:45.210772    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="eebf805781b95712cd1f5a6eed1eece5f8187856581b0d978c8a09ac2da331f7" exitCode=0
Oct 31 00:55:45.211220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:45.210814    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerDied","Data":"eebf805781b95712cd1f5a6eed1eece5f8187856581b0d978c8a09ac2da331f7"}
Oct 31 00:55:46.216134 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:46.216109    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:55:46.216625 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:46.216512    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"d90ccb9677456a0b9ba68fbe27d59684a52b4eef5b405867c1a71ba9389c813c"}
Oct 31 00:55:46.219720 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:46.219693    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" event={"ID":"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6","Type":"ContainerStarted","Data":"1f55906ffff95df1bac399bb9b0e4773481f86985748ef9c1ea00a483f804f3b"}
Oct 31 00:55:46.243451 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:46.243409    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d6xbt" podStartSLOduration=8.49580239 podStartE2EDuration="19.243399314s" totalImagesPullingTime="10.747596924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:46.242966498 +0000 UTC m=+19.645617622" watchObservedRunningTime="2026-10-31 00:55:46.243399314 +0000 UTC m=+19.646050439"
Oct 31 00:55:47.026941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:47.024499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:47.026941 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:47.024683    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:47.026941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:47.025373    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:47.026941 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:47.025502    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:47.026941 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:47.025556    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:47.026941 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:47.025666    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:48.227682 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.227504    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:55:48.228112 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.228087    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"193c98f839f98295303731b16d8ace35a3f6bfa38100b0bcc38a26ea6693dc7f"}
Oct 31 00:55:48.228400 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.228370    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:48.228496 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.228454    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:48.228496 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.228490    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:48.228740 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.228563    2574 scope.go:122] "RemoveContainer" containerID="dde362c46632d8a6ba801a42bdcaa8bed38d9c94ef2ec2738d5a40ef63980b28"
Oct 31 00:55:48.248120 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.248094    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:48.248252 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:48.248238    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:55:49.023110 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.023080    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:49.023312 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.023118    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:49.023312 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:49.023206    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:49.023312 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:49.023245    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:49.023312 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.023267    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:49.023458 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:49.023339    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:49.233025 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.233000    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:55:49.233377 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.233350    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" event={"ID":"90525854-b0fe-4589-8f36-df21ad285d0b","Type":"ContainerStarted","Data":"a12415d93f52ac9d535f888b1374e0691fbb67a92f964e7b493c8f9b35178d68"}
Oct 31 00:55:49.263387 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.263347    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c" podStartSLOduration=8.128486043 podStartE2EDuration="22.263337036s" totalImagesPullingTime="14.134850993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:55:49.262962109 +0000 UTC m=+22.665613235" watchObservedRunningTime="2026-10-31 00:55:49.263337036 +0000 UTC m=+22.665988160"
Oct 31 00:55:49.868541 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.868512    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6tsqv"]
Oct 31 00:55:49.868779 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.868634    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:49.868779 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:49.868723    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:49.871873 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.871847    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rs9gd"]
Oct 31 00:55:49.871997 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.871943    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:49.872044 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:49.872024    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:49.872707 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.872677    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7r985"]
Oct 31 00:55:49.872815 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:49.872803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:49.872940 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:49.872906    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:51.023720 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:51.023686    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:51.024187 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:51.023812    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:52.022669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:52.022622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:52.022847 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:52.022629    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:52.022847 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:52.022735    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:52.022932 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:52.022849    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:53.023309 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:53.023282    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:53.023669 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:53.023380    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:54.023219 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:54.023184    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:54.023410 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:54.023294    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:54.023410 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:54.023358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:54.023778 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:54.023467    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:55.026137 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:55.026072    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:55.026574 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:55.026164    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:56.023471 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:56.023398    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:56.023471 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:56.023435    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:56.023677 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:56.023522    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:56.023677 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:56.023568    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:57.025993 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:57.025847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:57.025993 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:57.026052    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:55:58.023183 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:58.023151    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:55:58.023183 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:58.023179    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:55:58.023376 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:58.023250    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:55:58.023414 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:58.023388    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:55:59.026366 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:55:59.026337    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:55:59.026746 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:55:59.026434    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:00.022911 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:00.022882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:00.022911 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:00.022914    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:00.023129 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.022997    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:00.023129 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.023103    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:00.699040 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:00.699006    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:00.699426 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.699151    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 00:56:00.699426 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.699209    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret podName:633346a3-a449-448c-8580-b87b22a05979 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:32.699196194 +0000 UTC m=+66.101847317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret") pod "global-pull-secret-syncer-6tsqv" (UID: "633346a3-a449-448c-8580-b87b22a05979") : object "kube-system"/"original-pull-secret" not registered
Oct 31 00:56:00.799346 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:00.799310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:00.799517 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:00.799354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:00.799517 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.799476    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 00:56:00.799517 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.799495    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 00:56:00.799517 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.799505    2574 projected.go:196] Error preparing data for projected volume kube-api-access-4wsdf for pod openshift-network-diagnostics/network-check-target-rs9gd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:56:00.799728 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.799546    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf podName:5e5138b9-caf5-4c38-adba-4d32a72b7da5 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:32.799533244 +0000 UTC m=+66.202184364 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsdf" (UniqueName: "kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf") pod "network-check-target-rs9gd" (UID: "5e5138b9-caf5-4c38-adba-4d32a72b7da5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 00:56:00.799728 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.799478    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:56:00.799728 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:00.799651    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs podName:41c16bf9-670b-4e0d-b009-8898ccad69b5 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:32.799633217 +0000 UTC m=+66.202284323 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs") pod "network-metrics-daemon-7r985" (UID: "41c16bf9-670b-4e0d-b009-8898ccad69b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 00:56:01.025660 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:01.025579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:01.025799 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:01.025686    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:02.022972 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:02.022938    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:02.023429 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:02.022976    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:02.023429 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:02.023041    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:02.023429 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:02.023097    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:03.025925 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:03.025902    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:03.026303 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:03.025989    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:04.023293 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:04.023259    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:04.023489 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:04.023259    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:04.023489 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:04.023384    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:04.023489 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:04.023468    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:05.023288 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:05.023254    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:05.023736 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:05.023367    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:06.023145 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:06.023105    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:06.023145 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:06.023150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:06.023720 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:06.023257    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:06.023720 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:06.023366    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:07.023754 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:07.023724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:07.024132 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:07.023803    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:08.023527 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:08.023495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:08.023721 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:08.023495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:08.023721 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:08.023590    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:08.023721 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:08.023712    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:09.023390 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:09.023358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:09.023545 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:09.023461    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:10.022940 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:10.022903    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:10.022940 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:10.022933    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:10.023346 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:10.023007    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:10.023346 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:10.023063    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:11.023227 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:11.023196    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:11.023697 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:11.023316    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:12.023211 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:12.023161    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:12.023211 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:12.023216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:12.023780 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:12.023309    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:12.023780 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:12.023422    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:13.023221 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:13.023191    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:13.023412 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:13.023308    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:14.023288 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:14.023251    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:14.023288 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:14.023276    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:14.023739 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:14.023355    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:14.023739 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:14.023464    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:15.023024 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:15.022987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:15.023214 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:15.023088    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rs9gd" podUID="5e5138b9-caf5-4c38-adba-4d32a72b7da5"
Oct 31 00:56:16.022952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.022929    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:16.023231 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:16.023039    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r985" podUID="41c16bf9-670b-4e0d-b009-8898ccad69b5"
Oct 31 00:56:16.023231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.023086    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:16.023231 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:16.023168    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tsqv" podUID="633346a3-a449-448c-8580-b87b22a05979"
Oct 31 00:56:16.297351 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.297276    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeReady"
Oct 31 00:56:16.297488 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.297390    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 00:56:16.352417 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.352381    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p2bzn"]
Oct 31 00:56:16.354247 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.354232    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.356113 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.356092    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p9js6"]
Oct 31 00:56:16.357514 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.357500    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372308    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372404    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372409    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372501    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372567    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-72bs2\""
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372566    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p5vm9\""
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372673    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p2bzn"]
Oct 31 00:56:16.372772 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.372717    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 00:56:16.373665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.373367    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p9js6"]
Oct 31 00:56:16.373665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.373384    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 00:56:16.415809 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.415779    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e575cbe0-30bf-4286-a382-ffe481a27c91-tmp-dir\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.415809 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.415808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c482845-4e0d-441c-aa58-44672e7f6bf3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.416000 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.415826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.416000 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.415847    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e575cbe0-30bf-4286-a382-ffe481a27c91-metrics-tls\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.416000 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.415976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c482845-4e0d-441c-aa58-44672e7f6bf3-crio-socket\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.416149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.416076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e575cbe0-30bf-4286-a382-ffe481a27c91-config-volume\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.416149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.416101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7jqp\" (UniqueName: \"kubernetes.io/projected/e575cbe0-30bf-4286-a382-ffe481a27c91-kube-api-access-t7jqp\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.416149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.416126    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjv87\" (UniqueName: \"kubernetes.io/projected/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-api-access-wjv87\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.416296 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.416230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c482845-4e0d-441c-aa58-44672e7f6bf3-data-volume\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.460909 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.460888    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r9zb2"]
Oct 31 00:56:16.462540 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.462526    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.469715 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.469690    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 00:56:16.469890 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.469787    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 00:56:16.471000 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.470972    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 00:56:16.471188 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.471176    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-58wnq\""
Oct 31 00:56:16.474025 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.474005    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r9zb2"]
Oct 31 00:56:16.517172 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c482845-4e0d-441c-aa58-44672e7f6bf3-crio-socket\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517330 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517205    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e575cbe0-30bf-4286-a382-ffe481a27c91-config-volume\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.517330 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7jqp\" (UniqueName: \"kubernetes.io/projected/e575cbe0-30bf-4286-a382-ffe481a27c91-kube-api-access-t7jqp\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.517330 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjv87\" (UniqueName: \"kubernetes.io/projected/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-api-access-wjv87\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517330 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517275    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-cert\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.517520 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c482845-4e0d-441c-aa58-44672e7f6bf3-data-volume\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517520 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517364    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c482845-4e0d-441c-aa58-44672e7f6bf3-crio-socket\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517520 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517510    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e575cbe0-30bf-4286-a382-ffe481a27c91-tmp-dir\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.517700 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c482845-4e0d-441c-aa58-44672e7f6bf3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517700 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517663    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517802 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbp2x\" (UniqueName: \"kubernetes.io/projected/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-kube-api-access-dbp2x\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.517802 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e575cbe0-30bf-4286-a382-ffe481a27c91-metrics-tls\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.517802 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517792    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c482845-4e0d-441c-aa58-44672e7f6bf3-data-volume\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.517923 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.517841    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e575cbe0-30bf-4286-a382-ffe481a27c91-tmp-dir\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.518137 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.518111    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e575cbe0-30bf-4286-a382-ffe481a27c91-config-volume\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.518199 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.518167    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.521362 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.521341    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e575cbe0-30bf-4286-a382-ffe481a27c91-metrics-tls\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.521445 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.521343    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c482845-4e0d-441c-aa58-44672e7f6bf3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.526774 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.526751    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7jqp\" (UniqueName: \"kubernetes.io/projected/e575cbe0-30bf-4286-a382-ffe481a27c91-kube-api-access-t7jqp\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.526846 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.526788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjv87\" (UniqueName: \"kubernetes.io/projected/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-api-access-wjv87\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.618807 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.618782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-cert\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.618937 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.618827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbp2x\" (UniqueName: \"kubernetes.io/projected/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-kube-api-access-dbp2x\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.621155 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.621134    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-cert\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.631551 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.631529    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbp2x\" (UniqueName: \"kubernetes.io/projected/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-kube-api-access-dbp2x\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.667433 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.667412    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:16.672201 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.672186    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:16.772012 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.771989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:16.953672 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.953588    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p2bzn"]
Oct 31 00:56:16.956064 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:16.956042    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p9js6"]
Oct 31 00:56:16.957874 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:16.957845    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c482845_4e0d_441c_aa58_44672e7f6bf3.slice/crio-37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5 WatchSource:0}: Error finding container 37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5: Status 404 returned error can't find the container with id 37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5
Oct 31 00:56:16.959065 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:16.959048    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode575cbe0_30bf_4286_a382_ffe481a27c91.slice/crio-0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e WatchSource:0}: Error finding container 0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e: Status 404 returned error can't find the container with id 0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e
Oct 31 00:56:17.026442 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.026419    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:17.031026 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.031000    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:56:17.031115 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.031104    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gczx4\""
Oct 31 00:56:17.041329 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.040903    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:56:17.047128 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.047067    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r9zb2"]
Oct 31 00:56:17.060909 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:17.060883    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319e5395_cce3_4e4d_9c3c_111e8eb1baaa.slice/crio-213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c WatchSource:0}: Error finding container 213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c: Status 404 returned error can't find the container with id 213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c
Oct 31 00:56:17.303177 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.303092    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r9zb2" event={"ID":"319e5395-cce3-4e4d-9c3c-111e8eb1baaa","Type":"ContainerStarted","Data":"213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c"}
Oct 31 00:56:17.304302 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.304274    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p9js6" event={"ID":"e575cbe0-30bf-4286-a382-ffe481a27c91","Type":"ContainerStarted","Data":"0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e"}
Oct 31 00:56:17.305630 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.305611    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2bzn" event={"ID":"4c482845-4e0d-441c-aa58-44672e7f6bf3","Type":"ContainerStarted","Data":"e8502ed0ffb3bc791b1dfdf7dca7a7dbbb1cd40f029655ec26a0f9fb784f7a14"}
Oct 31 00:56:17.305708 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:17.305648    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2bzn" event={"ID":"4c482845-4e0d-441c-aa58-44672e7f6bf3","Type":"ContainerStarted","Data":"37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5"}
Oct 31 00:56:18.023334 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.023255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:18.023334 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.023300    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:18.029210 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.029182    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:56:18.029589 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.029301    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-xb7hg\""
Oct 31 00:56:18.029589 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.029345    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:56:18.029589 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.029404    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89pfs\""
Oct 31 00:56:18.311085 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.310992    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2bzn" event={"ID":"4c482845-4e0d-441c-aa58-44672e7f6bf3","Type":"ContainerStarted","Data":"d655322092f7c7c7e354dfcf0ee44a241cb81146a0e79eaef965c9cff97349fd"}
Oct 31 00:56:18.489126 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.489092    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddx8v_ea64b244-e85f-494e-a824-9eba8ccfe22c/dns-node-resolver/0.log"
Oct 31 00:56:18.489126 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:18.489092    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddx8v_ea64b244-e85f-494e-a824-9eba8ccfe22c/dns-node-resolver/0.log"
Oct 31 00:56:19.316381 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.316281    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p9js6" event={"ID":"e575cbe0-30bf-4286-a382-ffe481a27c91","Type":"ContainerStarted","Data":"dc0e0481a00b519f9c30ab447d380087e55bdfc9171316aab8c5e946a3969fa0"}
Oct 31 00:56:19.316381 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.316347    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p9js6" event={"ID":"e575cbe0-30bf-4286-a382-ffe481a27c91","Type":"ContainerStarted","Data":"6196762f461e86ae0b7dbf7e623bdf9bcebe656ca5907ae64e9879e7b3d1f423"}
Oct 31 00:56:19.316893 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.316423    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:19.338408 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.338051    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-p9js6" podStartSLOduration=1.25470359 podStartE2EDuration="3.338033873s" totalImagesPullingTime="2.083330283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:19.335857271 +0000 UTC m=+52.738508396" watchObservedRunningTime="2026-10-31 00:56:19.338033873 +0000 UTC m=+52.740685003"
Oct 31 00:56:19.692867 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.692835    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hjrz2_f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a/node-ca/0.log"
Oct 31 00:56:19.693054 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.692835    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hjrz2_f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a/node-ca/0.log"
Oct 31 00:56:19.992487 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.992412    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"]
Oct 31 00:56:19.994304 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:19.994287    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:20.003555 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.003532    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 31 00:56:20.003678 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.003580    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f7djw\""
Oct 31 00:56:20.009898 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.009872    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"]
Oct 31 00:56:20.045964 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.045936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-mqpr5\" (UID: \"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:20.146286 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.146257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-mqpr5\" (UID: \"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:20.146412 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:20.146370    2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Oct 31 00:56:20.146462 ip-10-0-140-244 kubenswrapper[2574]: E1031 00:56:20.146422    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates podName:ab0a4936-76a5-40eb-a7bd-0822ea7c8de9 nodeName:}" failed. No retries permitted until 2026-10-31 00:56:20.646409594 +0000 UTC m=+54.049060697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates") pod "prometheus-operator-admission-webhook-6489d8bf89-mqpr5" (UID: "ab0a4936-76a5-40eb-a7bd-0822ea7c8de9") : secret "prometheus-operator-admission-webhook-tls" not found
Oct 31 00:56:20.246554 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.246482    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:20.319956 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.319911    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r9zb2" event={"ID":"319e5395-cce3-4e4d-9c3c-111e8eb1baaa","Type":"ContainerStarted","Data":"1a4c8df17f7610a83062782e089de14f08d5b358402891ca3a13e0f42a10b4bd"}
Oct 31 00:56:20.322003 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.321979    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2bzn" event={"ID":"4c482845-4e0d-441c-aa58-44672e7f6bf3","Type":"ContainerStarted","Data":"f5c20a703d03247050636b0e4ff1caac2c27d7b546fce1a22dc1d5fb02e5ec11"}
Oct 31 00:56:20.336918 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.336877    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r9zb2" podStartSLOduration=1.594149137 podStartE2EDuration="4.33686142s" totalImagesPullingTime="2.742712283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:20.336761116 +0000 UTC m=+53.739412255" watchObservedRunningTime="2026-10-31 00:56:20.33686142 +0000 UTC m=+53.739512547"
Oct 31 00:56:20.356179 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.356139    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p2bzn" podStartSLOduration=2.005554977 podStartE2EDuration="4.356127396s" totalImagesPullingTime="2.350572419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 00:56:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:20.355146782 +0000 UTC m=+53.757797932" watchObservedRunningTime="2026-10-31 00:56:20.356127396 +0000 UTC m=+53.758778521"
Oct 31 00:56:20.649939 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.649912    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-mqpr5\" (UID: \"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:20.652371 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.652352    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-mqpr5\" (UID: \"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:20.904132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:20.904040    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:21.184349 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:21.184266    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"]
Oct 31 00:56:21.187695 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:21.187659    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0a4936_76a5_40eb_a7bd_0822ea7c8de9.slice/crio-23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228 WatchSource:0}: Error finding container 23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228: Status 404 returned error can't find the container with id 23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228
Oct 31 00:56:21.324960 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:21.324922    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5" event={"ID":"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9","Type":"ContainerStarted","Data":"23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228"}
Oct 31 00:56:22.094175 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.094143    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f44b4d54f-scl9x"]
Oct 31 00:56:22.096114 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.096086    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.103735 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.103717    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 00:56:22.104759 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.104737    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 00:56:22.104913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.104895    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 00:56:22.104913 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.104903    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 00:56:22.105077 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.104937    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 00:56:22.105346 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.105329    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 00:56:22.105346 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.105338    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 00:56:22.105475 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.105342    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-55222\""
Oct 31 00:56:22.110112 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.110092    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f44b4d54f-scl9x"]
Oct 31 00:56:22.161236 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.161314 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161285    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.161365 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.161402 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.161444 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.161546 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.161582 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.161557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262151    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262299 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262299 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262299 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262299 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262299 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.262853 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.262824    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.263054 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.263033    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.263118 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.263053    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.263118 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.263090    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.264586 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.264564    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.264706 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.264690    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.271010 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.270992    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.330112 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.330085    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5" event={"ID":"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9","Type":"ContainerStarted","Data":"9461dc48b64de1991ad41a38ed45695feded8edc0493f85c4314da719a4a8b6e"}
Oct 31 00:56:22.330566 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.330540    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:22.335571 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.335543    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:22.349943 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.349871    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5" podStartSLOduration=2.3803737959999998 podStartE2EDuration="3.349859378s" totalImagesPullingTime="969.485582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:22.348875581 +0000 UTC m=+55.751526706" watchObservedRunningTime="2026-10-31 00:56:22.349859378 +0000 UTC m=+55.752510503"
Oct 31 00:56:22.408280 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.408247    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:22.680307 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:22.680280    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f44b4d54f-scl9x"]
Oct 31 00:56:22.683344 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:22.683318    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af5ed3a_6673_411b_87de_9ea59705bec6.slice/crio-1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95 WatchSource:0}: Error finding container 1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95: Status 404 returned error can't find the container with id 1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95
Oct 31 00:56:23.098373 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.098298    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"]
Oct 31 00:56:23.100806 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.100787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.109095 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.109076    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 00:56:23.109436 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.109421    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 00:56:23.109504 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.109438    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-z2mss\""
Oct 31 00:56:23.110554 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.110536    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 00:56:23.110554 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.110552    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 00:56:23.111058 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.111045    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 00:56:23.116847 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.116827    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"]
Oct 31 00:56:23.169458 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.169431    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.169458 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.169460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed848351-036d-461e-9295-e8eb410759cc-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.169686 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.169478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2xst\" (UniqueName: \"kubernetes.io/projected/ed848351-036d-461e-9295-e8eb410759cc-kube-api-access-v2xst\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.169686 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.169554    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.270157 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.270129    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.270273 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.270160    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed848351-036d-461e-9295-e8eb410759cc-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.270273 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.270181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2xst\" (UniqueName: \"kubernetes.io/projected/ed848351-036d-461e-9295-e8eb410759cc-kube-api-access-v2xst\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.270273 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.270260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.270871 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.270851    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed848351-036d-461e-9295-e8eb410759cc-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.272866 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.272842    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.272866 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.272855    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.278762 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.278743    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2xst\" (UniqueName: \"kubernetes.io/projected/ed848351-036d-461e-9295-e8eb410759cc-kube-api-access-v2xst\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.334702 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.334664    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f44b4d54f-scl9x" event={"ID":"6af5ed3a-6673-411b-87de-9ea59705bec6","Type":"ContainerStarted","Data":"1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95"}
Oct 31 00:56:23.411609 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.411581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:23.682498 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:23.682422    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"]
Oct 31 00:56:23.685166 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:23.685146    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded848351_036d_461e_9295_e8eb410759cc.slice/crio-94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e WatchSource:0}: Error finding container 94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e: Status 404 returned error can't find the container with id 94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e
Oct 31 00:56:24.340784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:24.340746    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7" event={"ID":"ed848351-036d-461e-9295-e8eb410759cc","Type":"ContainerStarted","Data":"94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e"}
Oct 31 00:56:25.345186 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:25.345107    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7" event={"ID":"ed848351-036d-461e-9295-e8eb410759cc","Type":"ContainerStarted","Data":"40ecbb16a4c4917ceb6003c36a17161645f4d31715fc5ca9fe1ef53249bfc3c3"}
Oct 31 00:56:25.345186 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:25.345157    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7" event={"ID":"ed848351-036d-461e-9295-e8eb410759cc","Type":"ContainerStarted","Data":"27a3bdb2c05f0bb662ff0ebf6392fe2fb448e0f91c2a87d2617d7943ffb779a5"}
Oct 31 00:56:25.368002 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:25.367950    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7" podStartSLOduration=1.158921256 podStartE2EDuration="2.367931229s" totalImagesPullingTime="1.209009973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:25.367406905 +0000 UTC m=+58.770058035" watchObservedRunningTime="2026-10-31 00:56:25.367931229 +0000 UTC m=+58.770582334"
Oct 31 00:56:26.844751 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.844715    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:56:26.846957 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.846922    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.857767 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.857726    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 00:56:26.859528 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.859506    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:56:26.895057 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895057 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895166    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895302 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895302 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.895411 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.895388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996086 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996139    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996171    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996421 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996379    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996556 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996642 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996589    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996705 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996705 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996848 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996831    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.996982 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.996959    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.997298 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.997276    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.999173 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.999141    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:26.999254 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:26.999147    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:27.004798 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.004776    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 00:56:27.006839 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.006816    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:27.007422 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.007400    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:27.160135 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.160094    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:27.451657 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.451633    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:56:27.454076 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:27.454047    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4641e5_a8d7_4cad_b10b_0989e80736ae.slice/crio-2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d WatchSource:0}: Error finding container 2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d: Status 404 returned error can't find the container with id 2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d
Oct 31 00:56:27.463286 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.463263    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"]
Oct 31 00:56:27.465992 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.465974    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.477572 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.477552    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 31 00:56:27.477753 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.477739    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 31 00:56:27.477830 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.477811    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-w8f42\""
Oct 31 00:56:27.478296 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.478265    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"]
Oct 31 00:56:27.500255 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.500228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.500377 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.500286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc7n2\" (UniqueName: \"kubernetes.io/projected/ba2e347b-434a-437a-bf36-85a85efd81bd-kube-api-access-jc7n2\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.500522 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.500499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.500666 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.500542    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2e347b-434a-437a-bf36-85a85efd81bd-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.528569 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.528542    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z9b9p"]
Oct 31 00:56:27.530660 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.530638    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.543920 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.543898    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"]
Oct 31 00:56:27.545993 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.545966    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.566464 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.566430    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 00:56:27.567149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.566871    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 00:56:27.567149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.566883    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 31 00:56:27.567149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.566900    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w6v48\""
Oct 31 00:56:27.567149 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.567074    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 31 00:56:27.569357 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.567542    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qgqml\""
Oct 31 00:56:27.569357 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.569122    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"]
Oct 31 00:56:27.601432 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc7n2\" (UniqueName: \"kubernetes.io/projected/ba2e347b-434a-437a-bf36-85a85efd81bd-kube-api-access-jc7n2\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.601432 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-textfile\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.601656 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601453    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.601656 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-wtmp\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.601656 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-root\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.601656 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601638    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-accelerators-collector-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.601840 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.601840 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.601840 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtbdk\" (UniqueName: \"kubernetes.io/projected/02e925ec-3898-42da-a8c2-5a1998d17c48-kube-api-access-dtbdk\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.601985 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.601985 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gc2n\" (UniqueName: \"kubernetes.io/projected/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-api-access-2gc2n\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.601985 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601922    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-tls\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.602132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.601985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2e347b-434a-437a-bf36-85a85efd81bd-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.602132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.602132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602052    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.602132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602069    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.602132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602087    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-sys\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.602132 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.602401 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-metrics-client-ca\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.602738 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.602712    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2e347b-434a-437a-bf36-85a85efd81bd-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.604450 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.604413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.604544 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.604529    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.606901 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.606884    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 00:56:27.607219 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.607198    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 31 00:56:27.615622 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.615583    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc7n2\" (UniqueName: \"kubernetes.io/projected/ba2e347b-434a-437a-bf36-85a85efd81bd-kube-api-access-jc7n2\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.703134 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtbdk\" (UniqueName: \"kubernetes.io/projected/02e925ec-3898-42da-a8c2-5a1998d17c48-kube-api-access-dtbdk\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703134 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.703340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gc2n\" (UniqueName: \"kubernetes.io/projected/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-api-access-2gc2n\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.703340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-tls\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703202    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.703340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703278    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-sys\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-metrics-client-ca\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-textfile\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-wtmp\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-root\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-accelerators-collector-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.703947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703815    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-wtmp\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703870    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-root\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.703947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703909    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-sys\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.704092 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.703956    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-textfile\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.704142 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.704118    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.704776 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.704683    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-accelerators-collector-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.704776 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.704715    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-metrics-client-ca\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.704931 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.704854    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.705097 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.705075    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.706485 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.706436    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-tls\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.706485 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.706444    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.706628 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.706547    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.707029 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.707008    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.717495 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.717467    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gc2n\" (UniqueName: \"kubernetes.io/projected/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-api-access-2gc2n\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:27.717906 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.717888    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtbdk\" (UniqueName: \"kubernetes.io/projected/02e925ec-3898-42da-a8c2-5a1998d17c48-kube-api-access-dtbdk\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.780971 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.780934    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:27.844825 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.844799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:27.857668 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:27.857641    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e925ec_3898_42da_a8c2_5a1998d17c48.slice/crio-35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953 WatchSource:0}: Error finding container 35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953: Status 404 returned error can't find the container with id 35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953
Oct 31 00:56:27.859447 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:27.859242    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:28.104681 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.104653    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"]
Oct 31 00:56:28.108298 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:28.108274    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2e347b_434a_437a_bf36_85a85efd81bd.slice/crio-2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462 WatchSource:0}: Error finding container 2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462: Status 404 returned error can't find the container with id 2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462
Oct 31 00:56:28.142845 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.142824    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"]
Oct 31 00:56:28.145583 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:28.145552    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee8d0cb_1686_4b84_ac7c_e07f8612d4ba.slice/crio-3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd WatchSource:0}: Error finding container 3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd: Status 404 returned error can't find the container with id 3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd
Oct 31 00:56:28.360890 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.360799    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l" event={"ID":"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba","Type":"ContainerStarted","Data":"3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd"}
Oct 31 00:56:28.362961 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.362930    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd" event={"ID":"ba2e347b-434a-437a-bf36-85a85efd81bd","Type":"ContainerStarted","Data":"9b0f5a996f216bea24b948890d93f38ca331598ffe34b69104a1f923dead290f"}
Oct 31 00:56:28.363060 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.362988    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd" event={"ID":"ba2e347b-434a-437a-bf36-85a85efd81bd","Type":"ContainerStarted","Data":"5bf3ea44c58f72ac8a841ec6a0a81a582e1ef647e2dd6745457debfa210e4eaf"}
Oct 31 00:56:28.363060 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.363017    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd" event={"ID":"ba2e347b-434a-437a-bf36-85a85efd81bd","Type":"ContainerStarted","Data":"2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462"}
Oct 31 00:56:28.364343 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.364318    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db8b47f85-pkjfx" event={"ID":"fe4641e5-a8d7-4cad-b10b-0989e80736ae","Type":"ContainerStarted","Data":"2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d"}
Oct 31 00:56:28.365738 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:28.365709    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z9b9p" event={"ID":"02e925ec-3898-42da-a8c2-5a1998d17c48","Type":"ContainerStarted","Data":"35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953"}
Oct 31 00:56:29.324742 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.324719    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:29.371819 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.371768    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z9b9p" event={"ID":"02e925ec-3898-42da-a8c2-5a1998d17c48","Type":"ContainerStarted","Data":"b46af0107e0065db33c7088103069582f682687c1b4e1b7f6f3d0503bd45b5c1"}
Oct 31 00:56:29.374297 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.373862    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f44b4d54f-scl9x" event={"ID":"6af5ed3a-6673-411b-87de-9ea59705bec6","Type":"ContainerStarted","Data":"134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c"}
Oct 31 00:56:29.434845 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.434794    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5f44b4d54f-scl9x" podStartSLOduration=1.75646768 podStartE2EDuration="7.43477604s" totalImagesPullingTime="5.67830836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:29.43395558 +0000 UTC m=+62.836606706" watchObservedRunningTime="2026-10-31 00:56:29.43477604 +0000 UTC m=+62.837427166"
Oct 31 00:56:29.600311 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.600282    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"]
Oct 31 00:56:29.604414 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.604390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.616192 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616157    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 31 00:56:29.616311 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616248    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d8z8j\""
Oct 31 00:56:29.616311 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616287    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 31 00:56:29.616430 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616342    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 31 00:56:29.616430 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616425    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3u451jijcguql\""
Oct 31 00:56:29.616519 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616453    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 31 00:56:29.616519 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.616480    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 31 00:56:29.626566 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.626543    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"]
Oct 31 00:56:29.723767 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.723725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.723911 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.723799    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-metrics-client-ca\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.723983 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.723943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.724023 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.723993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-tmp\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.724023 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.724011    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-grpc-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.724110 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.724026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8gkv\" (UniqueName: \"kubernetes.io/projected/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-kube-api-access-h8gkv\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.724110 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.724046    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.724183 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.724167    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.724231 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.724216    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825341 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-metrics-client-ca\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825428 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825387    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825428 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-tmp\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825502 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825433    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-grpc-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825502 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825448    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8gkv\" (UniqueName: \"kubernetes.io/projected/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-kube-api-access-h8gkv\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825502 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825466    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825502 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825496    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825639 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.825639 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.825550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.826184 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.826165    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-tmp\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.827167 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.827120    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-metrics-client-ca\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.828674 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.828591    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.830003 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.829425    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.830003 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.829724    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.830003 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.829954    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.831032 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.831004    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-grpc-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.831433 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.831390    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.840420 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.840399    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8gkv\" (UniqueName: \"kubernetes.io/projected/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-kube-api-access-h8gkv\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:29.926186 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:29.926157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:30.220396 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.220356    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"]
Oct 31 00:56:30.225868 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:30.225837    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af1e2d1_ebb9_4941_8b61_dd640def4b9d.slice/crio-5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff WatchSource:0}: Error finding container 5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff: Status 404 returned error can't find the container with id 5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff
Oct 31 00:56:30.379067 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.379034    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l" event={"ID":"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba","Type":"ContainerStarted","Data":"9097e145e52438fd4b9705056d72a17f7897011971efc9e55479214683ceca06"}
Oct 31 00:56:30.379529 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.379093    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l" event={"ID":"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba","Type":"ContainerStarted","Data":"d8c522dbd23fab16d2a684e50997453b57a8279af666f4f43ddc6125817aa773"}
Oct 31 00:56:30.379529 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.379123    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l" event={"ID":"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba","Type":"ContainerStarted","Data":"ac506d0f68b19f75089246362a6e974758e6f9bc6b28af2131eaf3ab358b3c74"}
Oct 31 00:56:30.381124 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.381102    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd" event={"ID":"ba2e347b-434a-437a-bf36-85a85efd81bd","Type":"ContainerStarted","Data":"722bf98896f03f7772915e1a1e71b64e3212dae399b52ed16ff8ca45625c5a00"}
Oct 31 00:56:30.382538 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.382518    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db8b47f85-pkjfx" event={"ID":"fe4641e5-a8d7-4cad-b10b-0989e80736ae","Type":"ContainerStarted","Data":"d20053c7b7f67ab34ecfd439975cfce7448853c7bafaaf15ad5e553a79719d7d"}
Oct 31 00:56:30.383640 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.383621    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff"}
Oct 31 00:56:30.385042 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.385024    2574 generic.go:356] "Generic (PLEG): container finished" podID="02e925ec-3898-42da-a8c2-5a1998d17c48" containerID="b46af0107e0065db33c7088103069582f682687c1b4e1b7f6f3d0503bd45b5c1" exitCode=0
Oct 31 00:56:30.385128 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.385111    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z9b9p" event={"ID":"02e925ec-3898-42da-a8c2-5a1998d17c48","Type":"ContainerDied","Data":"b46af0107e0065db33c7088103069582f682687c1b4e1b7f6f3d0503bd45b5c1"}
Oct 31 00:56:30.406444 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.406406    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l" podStartSLOduration=1.805340217 podStartE2EDuration="3.406393504s" totalImagesPullingTime="1.601053287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:30.404579859 +0000 UTC m=+63.807230984" watchObservedRunningTime="2026-10-31 00:56:30.406393504 +0000 UTC m=+63.809044665"
Oct 31 00:56:30.429434 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:30.429398    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6db8b47f85-pkjfx" podStartSLOduration=1.968545381 podStartE2EDuration="4.429386251s" totalImagesPullingTime="2.46084087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:30.428032294 +0000 UTC m=+63.830683432" watchObservedRunningTime="2026-10-31 00:56:30.429386251 +0000 UTC m=+63.832037432"
Oct 31 00:56:31.391084 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:31.391041    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z9b9p" event={"ID":"02e925ec-3898-42da-a8c2-5a1998d17c48","Type":"ContainerStarted","Data":"ea43c933e90b42506dc6b1c065d97170436b86ef7847cf524a390cd390294dfd"}
Oct 31 00:56:31.391502 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:31.391103    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z9b9p" event={"ID":"02e925ec-3898-42da-a8c2-5a1998d17c48","Type":"ContainerStarted","Data":"7ae7103291522621422825f6a6788369294d618cd4245cebfacceb470fc2f1af"}
Oct 31 00:56:31.443069 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:31.443008    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd" podStartSLOduration=3.24700716 podStartE2EDuration="4.44299385s" totalImagesPullingTime="1.19598669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:30.471964249 +0000 UTC m=+63.874615374" watchObservedRunningTime="2026-10-31 00:56:31.44299385 +0000 UTC m=+64.845644969"
Oct 31 00:56:31.443582 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:31.443551    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z9b9p" podStartSLOduration=2.953872873 podStartE2EDuration="4.443540062s" totalImagesPullingTime="1.384500756s" totalInitContainerRuntime="105.166433ms" isStatefulPod=true podCreationTimestamp="2026-10-31 00:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:31.441692193 +0000 UTC m=+64.844343320" watchObservedRunningTime="2026-10-31 00:56:31.443540062 +0000 UTC m=+64.846191221"
Oct 31 00:56:32.073235 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.073210    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6db4c97dcc-87jqk"]
Oct 31 00:56:32.075922 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.075903    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.089350 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.089330    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 31 00:56:32.089439 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.089380    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9qklh\""
Oct 31 00:56:32.089439 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.089410    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 31 00:56:32.090796 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.090772    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 00:56:32.090896 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.090839    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-57q8fd87sthu0\""
Oct 31 00:56:32.090896 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.090837    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 31 00:56:32.093481 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.093450    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6db4c97dcc-87jqk"]
Oct 31 00:56:32.148240 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-client-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.148323 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.148377 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148358    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzv22\" (UniqueName: \"kubernetes.io/projected/c1ada52a-5abd-4603-922d-cb20665f8a18-kube-api-access-gzv22\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.148419 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148401    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c1ada52a-5abd-4603-922d-cb20665f8a18-audit-log\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.148522 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-tls\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.148639 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-metrics-server-audit-profiles\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.148690 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.148675    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-client-certs\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.249585 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.249545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzv22\" (UniqueName: \"kubernetes.io/projected/c1ada52a-5abd-4603-922d-cb20665f8a18-kube-api-access-gzv22\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.249777 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.249590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c1ada52a-5abd-4603-922d-cb20665f8a18-audit-log\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.249855 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.249797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-tls\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.249945 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.249924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-metrics-server-audit-profiles\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.249997 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.249964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-client-certs\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.250051 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.250033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-client-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.250099 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.250047    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c1ada52a-5abd-4603-922d-cb20665f8a18-audit-log\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.250247 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.250226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.251279 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.251250    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-metrics-server-audit-profiles\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.251396 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.251375    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.252417 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.252394    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-tls\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.252633 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.252589    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-client-certs\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.252860 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.252836    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-client-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.261239 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.261221    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzv22\" (UniqueName: \"kubernetes.io/projected/c1ada52a-5abd-4603-922d-cb20665f8a18-kube-api-access-gzv22\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.306151 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.306075    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"]
Oct 31 00:56:32.308750 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.308731    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.326419 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.326395    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"]
Oct 31 00:56:32.350694 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.350668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.350829 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.350717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g95sl\" (UniqueName: \"kubernetes.io/projected/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-kube-api-access-g95sl\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.354090 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.354066    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 31 00:56:32.354090 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.354069    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7m7mt\""
Oct 31 00:56:32.387277 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.387241    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:32.396921 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.396894    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"36d6a0f59dfca20bdab69b35db8a1b10eadced7738380e516b0898bcc5151050"}
Oct 31 00:56:32.397258 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.396987    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"11d57b06c50a35ddb350385fd06698a276ba63555f73446ebff5b47e7ea9a2fd"}
Oct 31 00:56:32.397258 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.397016    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"12a43dc74a4b5d9bd45be94326b122f40ed91d56a041f327e4749b949cde63d3"}
Oct 31 00:56:32.409463 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.409439    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:32.409558 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.409495    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:32.415458 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.415435    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:32.451862 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.451234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.451862 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.451333    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95sl\" (UniqueName: \"kubernetes.io/projected/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-kube-api-access-g95sl\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.454076 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.454049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.462438 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.462410    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95sl\" (UniqueName: \"kubernetes.io/projected/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-kube-api-access-g95sl\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.619733 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.619702    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:32.665549 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.665481    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6db4c97dcc-87jqk"]
Oct 31 00:56:32.671845 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:32.671793    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ada52a_5abd_4603_922d_cb20665f8a18.slice/crio-75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2 WatchSource:0}: Error finding container 75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2: Status 404 returned error can't find the container with id 75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2
Oct 31 00:56:32.754743 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.754699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:32.757855 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.757833    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:56:32.768845 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.768816    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:32.784329 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.784299    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-68b5c76845-8wplb"]
Oct 31 00:56:32.787121 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.787103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.797292 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.797265    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 31 00:56:32.799594 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.799576    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 31 00:56:32.804732 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.804714    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 31 00:56:32.812799 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.812780    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68b5c76845-8wplb"]
Oct 31 00:56:32.855281 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855426 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms9ng\" (UniqueName: \"kubernetes.io/projected/d6f46adb-16dc-4058-bfd1-d33fca2885a3-kube-api-access-ms9ng\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855426 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855584 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855584 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855502    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-federate-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855704 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:32.855704 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855662    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-serving-certs-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855806 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855779    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855868 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-metrics-client-ca\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.855911 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.855872    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:32.859997 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.859969    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:56:32.860213 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.860198    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:56:32.869253 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.869219    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:56:32.871102 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.871049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:32.880625 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.880580    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:32.902935 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.902904    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 31 00:56:32.903074 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.903047    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2nprh\""
Oct 31 00:56:32.903133 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.903119    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 31 00:56:32.903260 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.903246    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 31 00:56:32.907908 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.905003    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"]
Oct 31 00:56:32.908379 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:32.908351    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9789f7f7_7d7d_4cd1_b74e_fd960dbb0283.slice/crio-843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235 WatchSource:0}: Error finding container 843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235: Status 404 returned error can't find the container with id 843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235
Oct 31 00:56:32.957256 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-serving-certs-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957409 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957320    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957409 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-metrics-client-ca\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957409 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957559 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9ng\" (UniqueName: \"kubernetes.io/projected/d6f46adb-16dc-4058-bfd1-d33fca2885a3-kube-api-access-ms9ng\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957559 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957449    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957559 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.957559 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.957513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-federate-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.959836 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.958019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-serving-certs-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.959836 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.958167    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-metrics-client-ca\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.959836 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.958224    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gczx4\""
Oct 31 00:56:32.960508 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.960486    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.960839 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.960821    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-federate-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.960952 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.960927    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.962947 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.962622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:32.965302 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.965275    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.966927 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.966876    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:32.968709 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:32.968681    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9ng\" (UniqueName: \"kubernetes.io/projected/d6f46adb-16dc-4058-bfd1-d33fca2885a3-kube-api-access-ms9ng\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:33.041039 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.041015    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-xb7hg\""
Oct 31 00:56:33.047914 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.047890    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89pfs\""
Oct 31 00:56:33.048949 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.048932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:33.055508 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.055491    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:33.102315 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.101999    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:33.248134 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.248109    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rs9gd"]
Oct 31 00:56:33.252866 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:33.252836    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e5138b9_caf5_4c38_adba_4d32a72b7da5.slice/crio-baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350 WatchSource:0}: Error finding container baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350: Status 404 returned error can't find the container with id baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350
Oct 31 00:56:33.330736 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.330705    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6tsqv"]
Oct 31 00:56:33.334166 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:33.334137    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633346a3_a449_448c_8580_b87b22a05979.slice/crio-ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312 WatchSource:0}: Error finding container ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312: Status 404 returned error can't find the container with id ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312
Oct 31 00:56:33.365677 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.365629    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7r985"]
Oct 31 00:56:33.367571 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:33.367548    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c16bf9_670b_4e0d_b009_8898ccad69b5.slice/crio-53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9 WatchSource:0}: Error finding container 53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9: Status 404 returned error can't find the container with id 53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9
Oct 31 00:56:33.401979 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.401919    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rs9gd" event={"ID":"5e5138b9-caf5-4c38-adba-4d32a72b7da5","Type":"ContainerStarted","Data":"baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350"}
Oct 31 00:56:33.403450 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.403431    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7r985" event={"ID":"41c16bf9-670b-4e0d-b009-8898ccad69b5","Type":"ContainerStarted","Data":"53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9"}
Oct 31 00:56:33.407631 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.407073    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"e738ff50019d6e22a8b8ffcc0cb00d9ea08a4b7277d25db24e9fd7b81fe4f641"}
Oct 31 00:56:33.407631 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.407164    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"ce054479e14d64ea772c91d4bc9824c8e711b5d3265e8c6ac72188bd434c8d9d"}
Oct 31 00:56:33.407631 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.407233    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:33.407631 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.407277    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" event={"ID":"9af1e2d1-ebb9-4941-8b61-dd640def4b9d","Type":"ContainerStarted","Data":"4c26038b1a5baf6c20dc96cdc9a0a86243fb7e501fc8e70b81670cb153a3845b"}
Oct 31 00:56:33.408318 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.408300    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6tsqv" event={"ID":"633346a3-a449-448c-8580-b87b22a05979","Type":"ContainerStarted","Data":"ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312"}
Oct 31 00:56:33.409571 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.409551    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2" event={"ID":"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283","Type":"ContainerStarted","Data":"843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235"}
Oct 31 00:56:33.410889 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.410866    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk" event={"ID":"c1ada52a-5abd-4603-922d-cb20665f8a18","Type":"ContainerStarted","Data":"75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2"}
Oct 31 00:56:33.416035 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.416017    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:33.423117 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.423097    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68b5c76845-8wplb"]
Oct 31 00:56:33.426689 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:33.426577    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f46adb_16dc_4058_bfd1_d33fca2885a3.slice/crio-033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9 WatchSource:0}: Error finding container 033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9: Status 404 returned error can't find the container with id 033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9
Oct 31 00:56:33.437125 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.437075    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7" podStartSLOduration=1.8379789739999999 podStartE2EDuration="4.437060815s" totalImagesPullingTime="2.599081841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:33.434571482 +0000 UTC m=+66.837222608" watchObservedRunningTime="2026-10-31 00:56:33.437060815 +0000 UTC m=+66.839711942"
Oct 31 00:56:33.868521 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.866116    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:56:33.871671 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.871211    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.878669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.877491    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 00:56:33.878669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.877576    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 00:56:33.878669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.877694    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 00:56:33.878669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.877943    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 00:56:33.878669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.878110    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 00:56:33.878669 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.878171    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 00:56:33.879493 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.879472    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 00:56:33.884115 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.883687    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 00:56:33.884115 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.883744    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-an58pn2d2p7d5\""
Oct 31 00:56:33.884115 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.883847    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 00:56:33.884115 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.884010    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vtd8r\""
Oct 31 00:56:33.884443 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.884425    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 00:56:33.889905 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.889881    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 00:56:33.900953 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.899833    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 00:56:33.900953 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.899884    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 00:56:33.900953 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.900025    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 00:56:33.902021 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.902001    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.969819    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.969866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.969903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.969934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.969975    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970008    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970150    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970184    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.970784 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970289    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.971665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.971665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.971665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970387    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:33.971665 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:33.970415    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071482    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.072196 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.071979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073628    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073714    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073773    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074122 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.073805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074491 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.074340    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.074491 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.074417    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.075171 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.075148    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.075271 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.075188    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.075435 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.075413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.076751 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.076707    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.079469 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.079438    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.080204 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.080156    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.082148 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.081347    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.082148 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.082088    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.086624 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.086566    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.102515 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.102494    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.104067 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.103503    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.104067 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.104025    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.104675 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.104631    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.105220 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.105171    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.105623 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.105558    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.106387 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.106345    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.106925 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.106887    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.114962 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.114932    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.222984 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.222835    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:34.419667 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.419623    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb" event={"ID":"d6f46adb-16dc-4058-bfd1-d33fca2885a3","Type":"ContainerStarted","Data":"033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9"}
Oct 31 00:56:34.548891 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:34.548814    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:56:34.551374 ip-10-0-140-244 kubenswrapper[2574]: W1031 00:56:34.551342    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db63efb_9319_4488_8589_2081d603b75e.slice/crio-d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc WatchSource:0}: Error finding container d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc: Status 404 returned error can't find the container with id d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc
Oct 31 00:56:35.252887 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.252827    2574 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:56:35.425085 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.425051    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc"}
Oct 31 00:56:35.426556 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.426528    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7r985" event={"ID":"41c16bf9-670b-4e0d-b009-8898ccad69b5","Type":"ContainerStarted","Data":"f073d1d79b72d84520ae41336bffd2692613e0e4cc0172e88590aba09709d723"}
Oct 31 00:56:35.427942 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.427913    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2" event={"ID":"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283","Type":"ContainerStarted","Data":"c7e971092221e023047237250b7ad4262dc6dc9bc9f18feca3bc0bb0b2437e3b"}
Oct 31 00:56:35.428340 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.428312    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:35.429494 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.429469    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk" event={"ID":"c1ada52a-5abd-4603-922d-cb20665f8a18","Type":"ContainerStarted","Data":"c101eef7cd62dafc4c3123feef7578c7307f5641c8276370c1361964e0d00269"}
Oct 31 00:56:35.433653 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.433622    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:35.449087 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.449047    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2" podStartSLOduration=1.405667773 podStartE2EDuration="3.449009763s" totalImagesPullingTime="2.04334199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:35.446833516 +0000 UTC m=+68.849484641" watchObservedRunningTime="2026-10-31 00:56:35.449009763 +0000 UTC m=+68.851660883"
Oct 31 00:56:35.469223 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:35.469167    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk" podStartSLOduration=1.235656499 podStartE2EDuration="3.469150792s" totalImagesPullingTime="2.233494293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:35.467777863 +0000 UTC m=+68.870429011" watchObservedRunningTime="2026-10-31 00:56:35.469150792 +0000 UTC m=+68.871801918"
Oct 31 00:56:36.438073 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:36.438027    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7r985" event={"ID":"41c16bf9-670b-4e0d-b009-8898ccad69b5","Type":"ContainerStarted","Data":"fe17c48454125e0df78bc07f516f97ad2020b7e2892cb5c7c46eeb1a71bfab1f"}
Oct 31 00:56:37.160854 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:37.160693    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:37.442963 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:37.442786    2574 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331" exitCode=0
Oct 31 00:56:37.443670 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:37.443639    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"}
Oct 31 00:56:37.503174 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:37.503118    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7r985" podStartSLOduration=68.60973035 podStartE2EDuration="1m10.50309224s" totalImagesPullingTime="1.89336189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:36.458426421 +0000 UTC m=+69.861077545" watchObservedRunningTime="2026-10-31 00:56:37.50309224 +0000 UTC m=+70.905743365"
Oct 31 00:56:39.427558 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:39.427528    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:41.482171 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:41.482131    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb" event={"ID":"d6f46adb-16dc-4058-bfd1-d33fca2885a3","Type":"ContainerStarted","Data":"46ca0f515a9e7d44974b2a7d4f477b9b2927ca02b3063ee18dcac8c263f3b8ae"}
Oct 31 00:56:41.482646 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:41.482200    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb" event={"ID":"d6f46adb-16dc-4058-bfd1-d33fca2885a3","Type":"ContainerStarted","Data":"e3d5180c11a1c6e4356b5154b2519cee42c1bcfadec89d25469c94b6e062d977"}
Oct 31 00:56:41.482646 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:41.482222    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb" event={"ID":"d6f46adb-16dc-4058-bfd1-d33fca2885a3","Type":"ContainerStarted","Data":"c3f4402ec7ce1cdea7051d830eb482027f9a78c8fb47dbbe4b04b154dfab87b7"}
Oct 31 00:56:41.522633 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:41.522561    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb" podStartSLOduration=1.9771350829999998 podStartE2EDuration="9.522544643s" totalImagesPullingTime="7.54540956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:41.521860528 +0000 UTC m=+74.924511654" watchObservedRunningTime="2026-10-31 00:56:41.522544643 +0000 UTC m=+74.925195770"
Oct 31 00:56:42.937267 ip-10-0-140-244 kubenswrapper[2574]: I1031 00:56:42.936782    2574 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:56:42.936961 ip-10-0-140-244 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 00:56:42.947056 ip-10-0-140-244 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 00:56:42.947257 ip-10-0-140-244 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 00:56:42.947311 ip-10-0-140-244 systemd[1]: kubelet.service: Consumed 4.534s CPU time.
Oct 31 00:56:42.958186 ip-10-0-140-244 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 00:56:43.049980 ip-10-0-140-244 kubenswrapper[8029]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:56:43.050694 ip-10-0-140-244 kubenswrapper[8029]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 00:56:43.050694 ip-10-0-140-244 kubenswrapper[8029]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:56:43.050694 ip-10-0-140-244 kubenswrapper[8029]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 00:56:43.053746 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053639    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.053746 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053735    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.053746 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053740    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.053746 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053745    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.053746 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053751    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.053746 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053756    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053761    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053767    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053770    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053773    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053775    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053778    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053780    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053793    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053796    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053799    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053802    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053804    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053807    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053810    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053812    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053815    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053817    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053820    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053823    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.054137 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053825    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053828    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053830    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053832    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053835    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053839    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053843    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053846    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053850    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053852    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053855    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053858    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053861    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053865    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053869    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053872    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053875    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053878    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053880    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053886    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.054758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053889    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053892    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053895    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053898    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053900    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053903    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053906    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053908    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053911    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053914    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053917    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053920    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053922    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053925    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053928    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053930    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053934    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053937    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053939    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.055391 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053942    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053945    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053947    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053949    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053952    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053955    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053960    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053962    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053965    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053968    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053970    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053973    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053975    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053978    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053981    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053984    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053986    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053989    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053992    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053995    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.056197 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.053998    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054001    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054003    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054006    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054009    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054011    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054014    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054016    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054019    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054021    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054024    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054027    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054029    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054032    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054034    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054037    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054039    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054042    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054047    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.056970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054050    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054052    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054056    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054939    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054950    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054955    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054959    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054964    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054969    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054975    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054979    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054985    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054989    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054994    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.054998    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055003    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055007    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055011    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055015    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055019    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.057743 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055024    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055028    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055033    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055037    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055042    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055046    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055050    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055054    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055059    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055063    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055068    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055077    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055082    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055086    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055089    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055091    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055096    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055100    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055104    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055109    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055116    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.058528 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055121    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055125    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055129    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055133    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055137    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055141    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055145    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055149    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055153    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055159    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055165    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055170    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055175    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055179    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055181    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055184    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055187    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055190    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055193    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.059302 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055195    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055198    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055201    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055205    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055209    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055212    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055214    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055217    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055220    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055223    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055225    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055228    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055230    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055234    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055237    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055239    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055242    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055244    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055247    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.060013 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055251    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055254    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055257    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055259    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055263    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055268    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055271    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055274    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055276    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055279    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055281    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055284    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055287    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055289    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055292    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055294    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055299    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055302    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055304    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.060825 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055307    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055309    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055312    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055315    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055317    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055320    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055322    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055326    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055329    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055331    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.055335    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056021    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056032    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056040    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056044    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056046    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056049    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056052    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056054    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.061589 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056057    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056060    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056063    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056066    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056071    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056076    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056084    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056091    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056097    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056102    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056106    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056109    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056112    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056115    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056117    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056120    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056123    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056128    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056133    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.062377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056137    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056142    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056148    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056153    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056156    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056159    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056161    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056165    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056168    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056170    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056173    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056176    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056178    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056181    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056184    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056188    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056192    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056197    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056200    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056203    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.063187 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056205    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056208    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056211    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056215    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056218    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056222    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056228    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056232    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056239    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056245    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056250    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056254    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056259    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056263    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056267    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056272    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056276    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056280    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056284    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056289    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.064011 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056293    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056298    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056303    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056308    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056312    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056317    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056322    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056328    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056332    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056337    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056341    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056344    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056349    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056353    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056357    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056364    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056369    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056373    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056378    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056383    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.064904 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056387    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056393    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056398    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056401    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056406    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056410    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056415    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056419    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056425    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056429    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056434    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056439    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056443    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056448    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056452    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056456    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056460    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056464    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.056468    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056633    8029 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056644    8029 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 00:56:43.065658 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056654    8029 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056661    8029 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056666    8029 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056671    8029 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056679    8029 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056685    8029 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056693    8029 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056698    8029 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056703    8029 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056708    8029 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056713    8029 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056718    8029 flags.go:64] FLAG: --cgroup-root=""
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056721    8029 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056725    8029 flags.go:64] FLAG: --client-ca-file=""
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056728    8029 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056732    8029 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056738    8029 flags.go:64] FLAG: --cluster-domain=""
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056742    8029 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056747    8029 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056752    8029 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056758    8029 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056763    8029 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056768    8029 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 00:56:43.066461 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056773    8029 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056778    8029 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056780    8029 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056783    8029 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056786    8029 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056790    8029 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056795    8029 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056801    8029 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056806    8029 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056810    8029 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056815    8029 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056819    8029 flags.go:64] FLAG: --enable-server="true"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056823    8029 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056830    8029 flags.go:64] FLAG: --event-burst="100"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056836    8029 flags.go:64] FLAG: --event-qps="50"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056841    8029 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056844    8029 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056847    8029 flags.go:64] FLAG: --eviction-hard=""
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056853    8029 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056857    8029 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056861    8029 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056866    8029 flags.go:64] FLAG: --eviction-soft=""
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056870    8029 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056875    8029 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056879    8029 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 00:56:43.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056883    8029 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056888    8029 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056892    8029 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056895    8029 flags.go:64] FLAG: --feature-gates=""
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056900    8029 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056904    8029 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056909    8029 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056915    8029 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056920    8029 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056925    8029 flags.go:64] FLAG: --help="false"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056929    8029 flags.go:64] FLAG: --hostname-override="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056934    8029 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056939    8029 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056944    8029 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056949    8029 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056952    8029 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056955    8029 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056959    8029 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056964    8029 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056968    8029 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056973    8029 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056978    8029 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056984    8029 flags.go:64] FLAG: --kube-reserved=""
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056988    8029 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 00:56:43.068339 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056993    8029 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.056998    8029 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057002    8029 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057006    8029 flags.go:64] FLAG: --lock-file=""
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057009    8029 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057014    8029 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057019    8029 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057027    8029 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057032    8029 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057036    8029 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057041    8029 flags.go:64] FLAG: --logging-format="text"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057045    8029 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057051    8029 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057055    8029 flags.go:64] FLAG: --manifest-url=""
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057061    8029 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057070    8029 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057075    8029 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057080    8029 flags.go:64] FLAG: --max-pods="110"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057085    8029 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057089    8029 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057094    8029 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057098    8029 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057103    8029 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057107    8029 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057112    8029 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 00:56:43.069347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057122    8029 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057127    8029 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057132    8029 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057138    8029 flags.go:64] FLAG: --pod-cidr=""
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057142    8029 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057147    8029 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057152    8029 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057157    8029 flags.go:64] FLAG: --port="10250"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057163    8029 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057168    8029 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-04527091bae342e72"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057173    8029 flags.go:64] FLAG: --qos-reserved=""
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057177    8029 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057180    8029 flags.go:64] FLAG: --register-node="true"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057183    8029 flags.go:64] FLAG: --register-with-taints=""
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057187    8029 flags.go:64] FLAG: --registry-burst="10"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057190    8029 flags.go:64] FLAG: --registry-qps="5"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057193    8029 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057196    8029 flags.go:64] FLAG: --reserved-memory=""
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057201    8029 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057206    8029 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057211    8029 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057216    8029 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057224    8029 flags.go:64] FLAG: --runonce="false"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057229    8029 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057235    8029 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057240    8029 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 00:56:43.070317 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057245    8029 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057268    8029 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057275    8029 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057281    8029 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057286    8029 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057291    8029 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057296    8029 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057301    8029 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057306    8029 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057312    8029 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057318    8029 flags.go:64] FLAG: --system-cgroups=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057324    8029 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057333    8029 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057338    8029 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057342    8029 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057349    8029 flags.go:64] FLAG: --tls-min-version=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057353    8029 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057358    8029 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057362    8029 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057366    8029 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057371    8029 flags.go:64] FLAG: --v="2"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057378    8029 flags.go:64] FLAG: --version="false"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057384    8029 flags.go:64] FLAG: --vmodule=""
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057391    8029 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.057397    8029 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 00:56:43.071296 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057586    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057594    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057631    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057637    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057641    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057645    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057650    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057655    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057660    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057664    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057670    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057673    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057676    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057678    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057681    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057685    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057689    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057692    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057696    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057699    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.072205 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057702    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057704    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057707    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057711    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057716    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057721    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057726    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057730    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057734    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057739    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057743    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057747    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057752    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057756    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057761    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057769    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057774    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057780    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057784    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.072970 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057789    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057792    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057796    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057800    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057805    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057809    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057814    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057819    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057822    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057825    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057829    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057835    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057839    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057844    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057849    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057853    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057858    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057862    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057866    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.073690 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057869    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057876    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057882    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057887    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057893    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057898    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057903    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057907    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057912    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057922    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057926    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057930    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057933    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057936    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057939    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057942    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057946    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057951    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057956    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.074435 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057960    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057965    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057970    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057975    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057979    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057983    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057987    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057992    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.057996    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058002    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058006    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058011    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058015    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058018    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058020    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058024    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058026    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058029    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058033    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058037    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.075076 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058041    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058045    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058058    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058063    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058067    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058072    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058077    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058081    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058085    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.058134    8029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058191    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058198    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058202    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058207    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058212    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058218    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.075570 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058223    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058228    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058232    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058236    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058241    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058246    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058250    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058254    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058258    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058262    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058266    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058270    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058274    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058279    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058283    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058288    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058293    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058298    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058307    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058314    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.076320 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058319    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058324    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058329    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058336    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058341    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058346    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058351    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058356    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058362    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058369    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058374    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058379    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058383    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058388    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058392    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058396    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058400    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058404    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058408    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.077055 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058413    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058417    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058422    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058426    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058430    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058435    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058439    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058443    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058447    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058451    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058456    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058462    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058468    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058472    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058477    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058482    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058486    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058491    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058495    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.077812 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058500    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058504    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058508    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058512    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058517    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058521    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058526    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058530    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058535    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058539    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058544    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058548    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058553    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058557    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058564    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058570    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058575    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058581    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058587    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.078581 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058592    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058613    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058619    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058623    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058627    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058637    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058644    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058649    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058652    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058657    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058662    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058667    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058673    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058677    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058682    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058687    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058691    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058696    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058701    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.079436 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058705    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058709    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058713    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.058719    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.067726    8029 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.067750    8029 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.067775    8029 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.067783    8029 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068136    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068145    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068149    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068154    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068160    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068164    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068167    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068170    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068173    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068176    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068179    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068182    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.080242 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068185    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068189    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068193    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068198    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068203    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068218    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068222    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068225    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068227    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068232    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068236    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068240    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068243    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068247    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068251    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068254    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068256    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068262    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068265    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.081071 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068267    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068270    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068273    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068278    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068283    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068288    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068292    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068295    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068298    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068301    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068306    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068309    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068312    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068314    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068317    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068320    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068322    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068326    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068330    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068350    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.081807 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068356    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068361    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068365    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068376    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068380    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068393    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068398    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068403    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068408    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068413    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068417    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068420    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068425    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068429    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068434    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068439    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068486    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068489    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068492    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.082559 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068495    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068499    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068504    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068509    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068513    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068518    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068522    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068525    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068528    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068530    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068536    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068539    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068541    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068551    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068554    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068557    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068561    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068566    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068570    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.083420 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068575    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068579    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068583    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068593    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068615    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068620    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068623    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068628    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068632    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068638    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068642    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068647    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068651    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068656    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068659    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068662    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.084377 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.068671    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.068730    8029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069708    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069721    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069727    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069737    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069741    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069744    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069750    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069752    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069756    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069764    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069767    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069770    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069774    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069779    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.085088 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069784    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069788    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069792    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069796    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069800    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069809    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069814    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069819    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069823    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069827    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069832    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069835    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069838    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069841    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069843    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069846    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069849    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069851    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069857    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.085758 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069860    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069862    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069865    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069868    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069871    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069873    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069876    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069880    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069884    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069895    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069898    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069904    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069907    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069909    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069912    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069915    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069918    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069923    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069927    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.086658 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069930    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069933    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069936    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069938    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069942    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069949    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069952    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069957    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.069961    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070040    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070044    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070048    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070052    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070056    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070060    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070064    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070067    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070075    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070079    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.087472 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070083    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070087    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070091    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070095    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070107    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070112    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070115    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070118    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070120    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070123    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070125    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070130    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070135    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070139    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070144    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070149    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070153    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070157    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070162    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070167    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.088358 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070172    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070177    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070182    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070192    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070197    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070201    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070206    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070210    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070214    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070218    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070222    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070227    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070230    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070234    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070239    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.070295    8029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 00:56:43.089294 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070487    8029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070499    8029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070505    8029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070509    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070514    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070520    8029 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070524    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070535    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070539    8029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070544    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070548    8029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070553    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070557    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070562    8029 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070568    8029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070573    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070578    8029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070582    8029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070588    8029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 00:56:43.090117 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070595    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070616    8029 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070621    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070626    8029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070631    8029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070634    8029 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070638    8029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070642    8029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070647    8029 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070652    8029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070657    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070662    8029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070667    8029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070673    8029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070675    8029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070685    8029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070690    8029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070693    8029 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070703    8029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070706    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 00:56:43.090949 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070709    8029 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070713    8029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070717    8029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070722    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070726    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070735    8029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070739    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070744    8029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070749    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070754    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070758    8029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070764    8029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070768    8029 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070773    8029 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070778    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070783    8029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070788    8029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070792    8029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070797    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070800    8029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 00:56:43.091844 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070803    8029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070807    8029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070814    8029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070818    8029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070822    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070827    8029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070830    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070834    8029 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070848    8029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070903    8029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070909    8029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070912    8029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070915    8029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070917    8029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070920    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070923    8029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070925    8029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070928    8029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070931    8029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 00:56:43.092823 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070934    8029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070938    8029 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070942    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070946    8029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070955    8029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070959    8029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070964    8029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070968    8029 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070972    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070977    8029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070980    8029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070983    8029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070985    8029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070988    8029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.070991    8029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071000    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071009    8029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071013    8029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071017    8029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071021    8029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 00:56:43.093879 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071025    8029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071031    8029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071043    8029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071046    8029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071049    8029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071052    8029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071054    8029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:43.071057    8029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.071327    8029 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.074156    8029 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.074855    8029 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.075115    8029 server.go:1014] "Starting client certificate rotation"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.075250    8029 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.075305    8029 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 00:50:26 +0000 UTC" deadline="2028-07-21 06:11:04.996941584 +0000 UTC"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.075340    8029 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15101h14m21.921603622s"
Oct 31 00:56:43.094831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.075577    8029 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:56:43.095545 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.077097    8029 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 00:56:43.095545 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.078908    8029 log.go:25] "Validated CRI v1 runtime API"
Oct 31 00:56:43.095545 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.081438    8029 log.go:25] "Validated CRI v1 image API"
Oct 31 00:56:43.095545 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.082409    8029 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 00:56:43.095545 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.086129    8029 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 94d64132-da43-4b26-95b1-356216f39f48:/dev/nvme0n1p3 9edc8f53-19b0-485c-b5b3-57909821b9c3:/dev/nvme0n1p4]
Oct 31 00:56:43.095907 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.086156    8029 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9/userdata/shm major:0 minor:429 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95/userdata/shm major:0 minor:264 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228/userdata/shm major:0 minor:251 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d/userdata/shm major:0 minor:289 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462/userdata/shm major:0 minor:302 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953/userdata/shm major:0 minor:304 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd/userdata/shm major:0 minor:308 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9/userdata/shm major:0 minor:423 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff/userdata/shm major:0 minor:359 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2/userdata/shm major:0 minor:399 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235/userdata/shm major:0 minor:403 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/85380a130db8fe5600daeda9f2fc60cfeb53199f87289a9ab6bef2d1bc18a471/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/85380a130db8fe5600daeda9f2fc60cfeb53199f87289a9ab6bef2d1bc18a471/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312/userdata/shm major:0 minor:421 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350/userdata/shm major:0 minor:417 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc/userdata/shm major:0 minor:460 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~projected/kube-api-access-dtbdk:{mountpoint:/var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~projected/kube-api-access-dtbdk major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:455 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~empty-dir/config-out major:0 minor:449 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~projected/kube-api-access-vcmvc:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~projected/kube-api-access-vcmvc major:0 minor:458 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~projected/tls-assets major:0 minor:454 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/config major:0 minor:448 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:457 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:456 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:452 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:459 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:453 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:450 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:451 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/web-config major:0 minor:455 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3005fee5-7711-481a-89dc-8f191ef548cd/volumes/kubernetes.io~projected/kube-api-access-m8hhn:{mountpoint:/var/lib/kubelet/pods/3005fee5-7711-481a-89dc-8f191ef548cd/volumes/kubernetes.io~projected/kube-api-access-m8hhn major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/319e5395-cce3-4e4d-9c3c-111e8eb1baaa/volumes/kubernetes.io~projected/kube-api-access-dbp2x:{mountpoint:/var/lib/kubelet/pods/319e5395-cce3-4e4d-9c3c-111e8eb1baaa/volumes/kubernetes.io~projected/kube-api-access-dbp2x major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/319e5395-cce3-4e4d-9c3c-111e8eb1baaa/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/319e5395-cce3-4e4d-9c3c-111e8eb1baaa/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41c16bf9-670b-4e0d-b009-8898ccad69b5/volumes/kubernetes.io~projected/kube-api-access-v884q:{mountpoint:/var/lib/kubelet/pods/41c16bf9-670b-4e0d-b009-8898ccad69b5/volumes/kubernetes.io~projected/kube-api-access-v884q major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41c16bf9-670b-4e0d-b009-8898ccad69b5/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/41c16bf9-670b-4e0d-b009-8898ccad69b5/volumes/kubernetes.io~secret/metrics-certs major:0 minor:408 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c482845-4e0d-441c-aa58-44672e7f6bf3/volumes/kubernetes.io~projected/kube-api-access-wjv87:{mountpoint:/var/lib/kubelet/pods/4c482845-4e0d-441c-aa58-44672e7f6bf3/volumes/kubernetes.io~projected/kube-api-access-wjv87 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c482845-4e0d-441c-aa58-44672e7f6bf3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/4c482845-4e0d-441c-aa58-44672e7f6bf3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e5138b9-caf5-4c38-adba-4d32a72b7da5/volumes/kubernetes.io~projected/kube-api-access-4wsdf:{mountpoint:/var/lib/kubelet/pods/5e5138b9-caf5-4c38-adba-4d32a72b7da5/volumes/kubernetes.io~projected/kube-api-access-4wsdf major:0 minor:409 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/633346a3-a449-448c-8580-b87b22a05979/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/633346a3-a449-448c-8580-b87b22a05979/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:407 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~projected/kube-api-access-tlc42:{mountpoint:/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~projected/kube-api-access-tlc42 major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/852b2fe1-0192-404e-9c39-e16b622a3b9f/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/852b2fe1-0192-404e-9c39-e16b622a3b9f/volumes/kubernetes.io~secret/agent-certs major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b4e40a1-88e5-493b-901d-668e910172cd/volumes/kubernetes.io~projected/kube-api-access-j7kl8:{mountpoint:/var/lib/kubelet/pods/8b4e40a1-88e5-493b-901d-668e910172cd/volumes/kubernetes.io~projected/kube-api-access-j7kl8 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volumes/kubernetes.io~projected/kube-api-access-tf85m:{mountpoint:/var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volumes/kubernetes.io~projected/kube-api-access-tf85m major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~empty-dir/tmp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~projected/kube-api-access-w92g4:{mountpoint:/var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~projected/kube-api-access-w92g4 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/volumes/kubernetes.io~projected/kube-api-access-g95sl:{mountpoint:/var/lib/kubelet/pods/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/volumes/kubernetes.io~projected/kube-api-access-g95sl major:0 minor:402 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:401 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~projected/kube-api-access-h8gkv:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~projected/kube-api-access-h8gkv major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:349 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6/volumes/kubernetes.io~projected/kube-api-access-k2ptv:{mountpoint:/var/lib/kubelet/pods/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6/volumes/kubernetes.io~projected/kube-api-access-k2ptv major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9/volumes/kubernetes.io~secret/tls-certificates major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~projected/kube-api-access-jc7n2:{mountpoint:/var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~projected/kube-api-access-jc7n2 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~projected/kube-api-access-gzv22:{mountpoint:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~projected/kube-api-access-gzv22 major:0 minor:398 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:397 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:396 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:392 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~projected/kube-api-access-ms9ng:{mountpoint:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~projected/kube-api-access-ms9ng major:0 minor:416 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:413 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:412 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:415 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:414 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e575cbe0-30bf-4286-a382-ffe481a27c91/volumes/kubernetes.io~projected/kube-api-access-t7jqp:{mountpoint:/var/lib/kubelet/pods/e575cbe0-30bf-4286-a382-ffe481a27c91/volumes/kubernetes.io~projected/kube-api-access-t7jqp major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e575cbe0-30bf-4286-a382-ffe481a27c91/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/e575cbe0-30bf-4286-a382-ffe481a27c91/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea64b244-e85f-494e-a824-9eba8ccfe22c/volumes/kubernetes.io~projected/kube-api-access-zthpv:{mountpoint:/var/lib/kubelet/pods/ea64b244-e85f-494e-a824-9eba8ccfe22c/volumes/kubernetes.io~projected/kube-api-access-zthpv major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~projected/kube-api-access-v2xst:{mountpoint:/var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~projected/kube-api-access-v2xst major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~projected/kube-api-access-2gc2n:{mountpoint:/var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~projected/kube-api-access-2gc2n major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a/volumes/kubernetes.io~projected/kube-api-access-4ftlv:{mountpoint:/var/lib/kubelet/pods/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a/volumes/kubernetes.io~projected/kube-api-access-4ftlv major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb321839-52ad-47ca-b4d6-b30b18970649/volumes/kubernetes.io~projected/kube-api-access-g2j2l:{mountpoint:/var/lib/kubelet/pods/fb321839-52ad-47ca-b4d6-b30b18970649/volumes/kubernetes.io~projected/kube-api-access-g2j2l major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~projected/kube-api-access-xllqx:{mountpoint:/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~projected/kube-api-access-xllqx major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:283 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/2df2df7da285f860795581efcccc3a75315173c3c236545850c3965d7ac1286e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/6078603731e3ba5322fd46036eefcec56487307f8dd47091fc3dd3661cd9027b/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/62068429fff9a9d84df5fa7976da13570dfddec9fc19cb8f5d7ae1e92f8d4c52/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/3498f27ebaf7cf9abd8c6ade242da483ecc0160952a4d9d3be00dead8bab7a79/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/08fc4161a0156238b1c932e4770aacadc6bbc2fc7ef550d3fa3f348fc246931f/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/ecd2501c2307d6b2712d847710425ca25b6ad0fdb9a8374820a8a7b9ac3407ef/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/31f4b40c8a59ed974690c8577cc57755136d58a123939fb9e3e5b5c4bcb2461b/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/c5d8a9691fd3aa66f61574fc92b1a7ba47f7f93db3646adfbed3b1d0526b3960/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/290f9ff021501a560d96e2dccf91c1377bcc93b077d71050c4a59e32913fe36b/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/5f7e13ef0230e3ddb271cbf83faa9923f7479deefa67654b1bb5fafdfc7eb549/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-140:{mountpoint:/var/lib/containers/storage/overlay/a7c9e2803b4c4faea552eb8413e15e69e32815b5443ef250e52b5d238f3e837e/merged major:0 minor:140 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/b95562fab27016a9de657a094ed4045f631005edaa9518e894d552f4036b2fe6/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/6c4bb344369aace87b8557351b188bb3911e5fb56ed924f9ba1e968477c03cb9/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/f80e9f8a3163a716863c70eea4d9b754ddaf9369bad9480934a278f8c349056e/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/d3466a8c88204fd9ff103cd917a7d87235693134bd72ccdfc2aea9bd7a0a3bd5/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/362fc3682073778bad25cf09d9d6c5e2279083071f22bd309b1c1e61f0ab310b/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/c6d9b78e37ad49cb0fc5a5ffb55f5b11d23146ac02f74c46d3029699bbdd01ee/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/b2a05b088f3d31a4d8e8a155ca4532ccc89257ed79380861af4aa8a746a824ca/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/37a93f5e25ea810714222b396aa7685ecff582e209779a66ad8d88c549c1f98c/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c9db1a09016afdda83214f5936a7d3f63905b5e7868cc1ee368283b6af1f8969/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/2304be607f54c7357d98a79882f8d8a85ae118b9ece0c2a4d7bb58916bcba32c/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/8f23bec69a36f7be890af17eb163e176a94fdb655190e5b353965539e4868b6f/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/d0708b630a1170e8cc96e8659f3d65ceb19b4365fa15ae4f5bc958399bf7e8a0/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/7d45f9005903ef2b04888ba046358975e488aaeeaa8576ee5c6134858df98be4/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/cfa4dba2c608b67e76c2fedc941e8f090a7003b0bc90b14152fadd54cb4fb18f/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/d372667ff746cb88f75164ed0c1da96e1d5685e9a1a316172c00fc6af7158828/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/4f10981462e439db1bfe0a6494648fd91c073261fbab5e379badd3c52ee72643/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/bd149851e81bd1cc96748b5570e84c7d1584ee5cafe50e976e43ba70beb80d9a/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/7adb96ded446458c4140ecb1b64861b7b69782fff7a9a0c94eb888d26ae830c5/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/ffae107572ea68c3605e2abbd45398f70e4fe425f70d9b7cae65594e63798aa1/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/7c6146ab4824a4a37615f58d9862ec0252aed40389b452baaeadda6f8d5965e7/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/b006941ffe86c2046056233dd35e14e400f6e3d86759f3cd83e390c95c8db19e/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/3bd567529033e7f265b10912f3713b227b7c461efe8a548dc753cc0907a6fc21/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/fbf6f39d0b617390b6d2bec80c24f06475d29fab24f94fe7c330c96c601f82af/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/d7195fbbc87cf6b87c3600df95939176bc5a0782afa794cfa6c58aa22eddff38/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/947a1e0ba4d4317f40f5070d8135fafce57a60c1ab2ab0031ee396daadd04d8a/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/3de92b8a4b9bffe5c43fd37cf1cd7d8701bb6fb0b680a49f904cc2ab040df7c5/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/2caaef3402a08e34898cb356a9fd313d366579df5825cfd8ad62cc618711f33c/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/e601c020cca2c5525e1125607d23e621146fabed16800ac99bb9794e2b8128c1/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/9757f967c5e86d9aa0625ece3c7c6dabbf87d6b7ecc2a658ef6d9de7088ba3d5/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/6ab4b95ced475b68545d8a376bbb0bc0e8dd31b6046631721df328f0221102b4/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/2a1807c1c85530f0712b7461cd4b6d28dcba70e15bd21ae263a5ce5df8e4f066/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/0cfe292348af1264acaf488181a561967c0607f291cb869db0b6ef549e15d2cd/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/d38dee31b810c2c5d6992e8172cc5aa60f3f49c9f9ae09579d050955c8158730/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/b0a8b3dada3cb118fadf51dd44245323e89cc81572390fe9ec76cd39b9ee0047/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/916886297eb867f0d9411de8e76f6dfb09f5e7e7d1d7110716e0f3e741313c18/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/fa5865cc7d25b8e77620089c79f0a0d61de4aef51df286e9e37cb7dc29b27dd3/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/1a8813e14b573a10f8d1fcff626c54bf7a58a0dffcbae319436dd22a8bc3a7f0/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/4397e3f15199caaa97bda7827576c21bc32fb979f7f9ff01991e63a1ccfc68ff/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/5df33a7cbe63d389e60b35badd0ddfc65dd3cb127ec76e962d5d3c4fe420206e/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/801cb6e9b2909196563f5f46fb39b94b5f7d88f63f4ef81ee858292ed27aff5b/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/2edab1020cca9dee2d33b59a3417c0552fc6942ab890ae1be7a62ab3240285c0/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/afe6cb47a594565a0a34eff42946884f722bf6b8411cfe687e15a8e4a50e8140/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/e8dc4a465f79966d85b065a47b1586e179669527e93cbb44ff8468e5994e27af/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/96959f2f383604b06522baeb70fa4bfddb3cf32de9d4f6568adebf5470b66ed2/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-410:{mountpoint:/var/lib/containers/storage/overlay/2a75acff9040105240ebc013085eed23b2a03cd2a7b09c21e4a4ac94fb3c6542/merged major:0 minor:410 fsType:overlay blockSize:0} overlay_0-419:{mountpoint:/var/lib/containers/storage/overlay/a4500846281ac1640efcd2bd827fbb2c5b64361b560c09ecfa600cce9c8f2e47/merged major:0 minor:419 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/8786df4ad6d9c7a90316a863e2250f58b80b34a52a183d517585f1e2c627bdd1/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-432:{mountpoint:/var/lib/containers/storage/overlay/1619294b2c3d56a544ae46d970fb0afb2bb8cfd60a003881ada8fc8b7f7a7914/merged major:0 minor:432 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/a0d1aa581b0fafce77254c60db8fbb1f1676de79b5b6d557a5e269c1739d51e9/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-439:{mountpoint:/var/lib/containers/storage/overlay/7966e57ebceef9fba346680375164835d2c41a5625ad7acfc5f55d7efa07fc42/merged major:0 minor:439 fsType:overlay blockSize:0} overlay_0-444:{mountpoint:/var/lib/containers/storage/overlay/6561937c794fadec4acf936f1044c1edafc512f9c19e6072e810d38035ad53cc/merged major:0 minor:444 fsType:overlay blockSize:0} overlay_0-446:{mountpoint:/var/lib/containers/storage/overlay/6ab0406d58c5191980620583358f3136173db6ba16306b8a8b05bc7237127ebf/merged major:0 minor:446 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/86b436cab007eb8fad44d8f0b43ed2cddf91065d820ed0ee3ee601a3b4a75f27/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-464:{mountpoint:/var/lib/containers/storage/overlay/1e3102f9a96b53a4812aba40d4787a7c33e79288320a73bf19a486da12676e5c/merged major:0 minor:464 fsType:overlay blockSize:0} overlay_0-466:{mountpoint:/var/lib/containers/storage/overlay/673f787b6a907db9ee127548c4235f09b88c5f9c0504a644e16091abc2c8b763/merged major:0 minor:466 fsType:overlay blockSize:0} overlay_0-471:{mountpoint:/var/lib/containers/storage/overlay/e6aeaec4dc49ceb3811f291e80716cf59cf5256c4126ec114d6d5e0631d83a9d/merged major:0 minor:471 fsType:overlay blockSize:0} overlay_0-478:{mountpoint:/var/lib/containers/storage/overlay/27d984848e425e11cecbaf1ecd25e158f08147bdb878e22c9423d1659ad5a604/merged major:0 minor:478 fsType:overlay blockSize:0} overlay_0-484:{mountpoint:/var/lib/containers/storage/overlay/db9e554ce0ce403deeb512fe7a92d0678f598dc20d573261813b2a4c94537bc3/merged major:0 minor:484 fsType:overlay blockSize:0} overlay_0-489:{mountpoint:/var/lib/containers/storage/overlay/227758e3f88bb43f3f3fdf5266c707cdac594b3fb8fabee9e0661561945dc7ce/merged major:0 minor:489 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/031a675e5b4599605395795969fbf6d64e180c172947720c5aaf02a509721f92/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-495:{mountpoint:/var/lib/containers/storage/overlay/58e4d519507a0dd4d92dc404bc209212b439b10e8717b27321359ac52f32371b/merged major:0 minor:495 fsType:overlay blockSize:0} overlay_0-500:{mountpoint:/var/lib/containers/storage/overlay/4d577b025a9f629e69f501357b34059c4fd1f3320b4cda55a0c1be24163a63ee/merged major:0 minor:500 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/a935da1d8960b9fe52e4826e83c1168483e64cd5eada9c203ac679129b099549/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/39943c6aa6eb01358736c76893558fab9cfeb9412f24e5c68286178b55355784/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/1ac9d6aee3962991bcde82ca42ca2e6784cc1f2cd67866fc59f8a3078ca90402/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/b2721f7e6bffa094b4a168ce69877e09a8090357095d107af077f539fa998186/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/72f5b4c8761a9c3cc1db516bfbe66ed6508b831ef4b345d49897c4d33794c7f6/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/87054d7f93c3ae3527dff8b796e90ad6e6ed7d2db528fc7e7a68f68a684b43bf/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/c9df4b01110151e9c0f7fc81166e66d170fe6afecb637eb2ef7f65d1eb78ce24/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/817515818a0239f8fd3954f5c18acefec0027aaa31e7ccfa29b94cb4e108052f/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/aa192997daf23580ac5ce2aef1eef77ef77fec159d4eac74566b4fff105da9a9/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 00:56:43.098447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.098018    8029 manager.go:217] Machine: {Timestamp:2026-10-31 00:56:43.097530229 +0000 UTC m=+0.084642195 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099776 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b06d667be95542c9404c2a5fd1d05 SystemUUID:ec2b06d6-67be-9554-2c94-04c2a5fd1d05 BootID:9d078ce9-753b-4550-a2c1-36c362e2393d Filesystems:[{Device:/run/containers/storage/overlay-containers/f5383294eec76a52335d8ec7bb148b3e7628d9a61da52dbe28188fbc18288d8a/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:299 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:344 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-446 DeviceMajor:0 DeviceMinor:446 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:455 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/319e5395-cce3-4e4d-9c3c-111e8eb1baaa/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d/userdata/shm DeviceMajor:0 DeviceMinor:289 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/baa63197856687efa5e43af85cd81219a6a29e2700830bc79869b371d237e350/userdata/shm DeviceMajor:0 DeviceMinor:417 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-471 DeviceMajor:0 DeviceMinor:471 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-484 DeviceMajor:0 DeviceMinor:484 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~projected/kube-api-access-w92g4 DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cff128dde4d8460b18f3d8dedbd90262fecae74b271568dbcde62a3746795a7c/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~projected/kube-api-access-xllqx DeviceMajor:0 DeviceMinor:288 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:345 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:348 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-439 DeviceMajor:0 DeviceMinor:439 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e4be7509a4ba3f59528905c221b39a8c685139e7d5bdef94701dc8f7e33de88e/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:262 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:298 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/35600159e0df18626129eb95cc5f7172bfb9af2dcf5348972564635a028a1953/userdata/shm DeviceMajor:0 DeviceMinor:304 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94561f7713dd00d887ab528ba5ab08a775cb6080dd39ece3f98de11650cad22e/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~projected/kube-api-access-gzv22 DeviceMajor:0 DeviceMinor:398 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/volumes/kubernetes.io~projected/kube-api-access-g95sl DeviceMajor:0 DeviceMinor:402 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/843e541ef8e254924ad5d8ef508c0ac5ea4436f0bff8a4ae3934076b524a8235/userdata/shm DeviceMajor:0 DeviceMinor:403 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-419 DeviceMajor:0 DeviceMinor:419 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:448 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~projected/kube-api-access-vcmvc DeviceMajor:0 DeviceMinor:458 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4c482845-4e0d-441c-aa58-44672e7f6bf3/volumes/kubernetes.io~projected/kube-api-access-wjv87 DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:413 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab2e8504c22308e3195ea7319e5fa9366e3095c5c18e21935d3a83ab17b3e312/userdata/shm DeviceMajor:0 DeviceMinor:421 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-464 DeviceMajor:0 DeviceMinor:464 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:412 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:456 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-478 DeviceMajor:0 DeviceMinor:478 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-500 DeviceMajor:0 DeviceMinor:500 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:294 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/633346a3-a449-448c-8580-b87b22a05979/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:407 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/e575cbe0-30bf-4286-a382-ffe481a27c91/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/23e26fbe112966cdff24f831cb175020539cd88f8db993d58b25e773cb7fd228/userdata/shm DeviceMajor:0 DeviceMinor:251 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:296 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a/volumes/kubernetes.io~projected/kube-api-access-4ftlv DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~projected/kube-api-access-jc7n2 DeviceMajor:0 DeviceMinor:295 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:401 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/53b81f3152a17c41b4350d8aaf4a49ece811119a14d3e34c4cebf9cb7b676be9/userdata/shm DeviceMajor:0 DeviceMinor:423 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volumes/kubernetes.io~projected/kube-api-access-tf85m DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:268 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:452 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-489 DeviceMajor:0 DeviceMinor:489 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~projected/kube-api-access-v2xst DeviceMajor:0 DeviceMinor:270 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:457 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7b11b0930ce8ef667dc59b5b367a7e469064c01220ddd102ea4d152fc13ffb3a/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4c482845-4e0d-441c-aa58-44672e7f6bf3/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5840307fcd72a4f42f0f8ebec01f3a2ca24953131799c09154aa8065f0ba40ff/userdata/shm DeviceMajor:0 DeviceMinor:359 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-495 DeviceMajor:0 DeviceMinor:495 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b4e40a1-88e5-493b-901d-668e910172cd/volumes/kubernetes.io~projected/kube-api-access-j7kl8 DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/852b2fe1-0192-404e-9c39-e16b622a3b9f/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7c8e7a7ba74e6fe9572164460604f1da0447941ef2872646b5c35719d59c0b41/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:283 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:346 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:246 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/033cbce38ce8d07bb4a481470ab7bca7d980e637e2f788fe9b287caf37e75da9/userdata/shm DeviceMajor:0 DeviceMinor:429 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fb321839-52ad-47ca-b4d6-b30b18970649/volumes/kubernetes.io~projected/kube-api-access-g2j2l DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc/userdata/shm DeviceMajor:0 DeviceMinor:460 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3005fee5-7711-481a-89dc-8f191ef548cd/volumes/kubernetes.io~projected/kube-api-access-m8hhn DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-140 DeviceMajor:0 DeviceMinor:140 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:347 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/75363760a3df34a67c4388af3a31505a769a58633982bce234273f2d224d1fa2/userdata/shm DeviceMajor:0 DeviceMinor:399 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41c16bf9-670b-4e0d-b009-8898ccad69b5/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:408 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:349 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2c435978759ac501e3aa41595e3ff3694b9a81473b20364b20c5c58abde16462/userdata/shm DeviceMajor:0 DeviceMinor:302 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e8e4a76f3052fb4cd7f8433063ddf2e61d6b9417b50f9331c6aefb9e04cf8fdb/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:396 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:453 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:459 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41c16bf9-670b-4e0d-b009-8898ccad69b5/volumes/kubernetes.io~projected/kube-api-access-v884q DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/85380a130db8fe5600daeda9f2fc60cfeb53199f87289a9ab6bef2d1bc18a471/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/319e5395-cce3-4e4d-9c3c-111e8eb1baaa/volumes/kubernetes.io~projected/kube-api-access-dbp2x DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95/userdata/shm DeviceMajor:0 DeviceMinor:264 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ea64b244-e85f-494e-a824-9eba8ccfe22c/volumes/kubernetes.io~projected/kube-api-access-zthpv DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:454 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37bb6d6520b33fa9d0321b7f19d1f6719649d5a111fe60d70efcb3fee12679e5/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0299bf1f8515292cca8cbcb673bb1508932b62568f1c1ea45ec2a1874d899c2e/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/02e925ec-3898-42da-a8c2-5a1998d17c48/volumes/kubernetes.io~projected/kube-api-access-dtbdk DeviceMajor:0 DeviceMinor:301 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:449 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/90525854-b0fe-4589-8f36-df21ad285d0b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:392 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ba2e347b-434a-437a-bf36-85a85efd81bd/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:293 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:297 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c61dd0f61e94f27628a139b18f38958129b8bc6a4dc296a1d5bc1416999d3bd/userdata/shm DeviceMajor:0 DeviceMinor:308 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ed848351-036d-461e-9295-e8eb410759cc/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:269 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/volumes/kubernetes.io~projected/kube-api-access-2gc2n DeviceMajor:0 DeviceMinor:300 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9af1e2d1-ebb9-4941-8b61-dd640def4b9d/volumes/kubernetes.io~projected/kube-api-access-h8gkv DeviceMajor:0 DeviceMinor:350 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c1ada52a-5abd-4603-922d-cb20665f8a18/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:397 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:450 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-466 DeviceMajor:0 DeviceMinor:466 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:257 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-410 DeviceMajor:0 DeviceMinor:410 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:414 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:455 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6/volumes/kubernetes.io~projected/kube-api-access-k2ptv DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-432 DeviceMajor:0 DeviceMinor:432 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/213871022d3b3128bf5bd342715b926b8414694bc96d388991deb3b9565ccd2c/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes/kubernetes.io~projected/kube-api-access-tlc42 DeviceMajor:0 DeviceMinor:263 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:415 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:451 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/83cdbe04b428d422c55e36b2fb7e5a50151f35d78241bcf8d9374aaa216adfc1/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e575cbe0-30bf-4286-a382-ffe481a27c91/volumes/kubernetes.io~projected/kube-api-access-t7jqp DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5e5138b9-caf5-4c38-adba-4d32a72b7da5/volumes/kubernetes.io~projected/kube-api-access-4wsdf DeviceMajor:0 DeviceMinor:409 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d6f46adb-16dc-4058-bfd1-d33fca2885a3/volumes/kubernetes.io~projected/kube-api-access-ms9ng DeviceMajor:0 DeviceMinor:416 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-444 DeviceMajor:0 DeviceMinor:444 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0299bf1f8515292 MacAddress:d2:3d:9a:53:56:98 Speed:10000 Mtu:8901} {Name:033cbce38ce8d07 MacAddress:da:56:97:83:b2:ba Speed:10000 Mtu:8901} {Name:1ab0b0b0c6609ff MacAddress:a2:6b:ba:57:bb:03 Speed:10000 Mtu:8901} {Name:213871022d3b312 MacAddress:ae:9c:ab:9f:01:f2 Speed:10000 Mtu:8901} {Name:23e26fbe112966c MacAddress:4e:62:21:51:89:b5 Speed:10000 Mtu:8901} {Name:2941b4baa1289c4 MacAddress:12:06:b8:cb:78:45 Speed:10000 Mtu:8901} {Name:2c435978759ac50 MacAddress:62:c0:d7:5c:94:61 Speed:10000 Mtu:8901} {Name:37bb6d6520b33fa MacAddress:9a:3a:a0:4a:e8:f9 Speed:10000 Mtu:8901} {Name:3c61dd0f61e94f2 MacAddress:5e:23:d4:35:d4:b4 Speed:10000 Mtu:8901} {Name:53b81f3152a17c4 MacAddress:16:57:45:e2:e2:18 Speed:10000 Mtu:8901} {Name:5840307fcd72a4f MacAddress:c6:c7:c4:73:c9:f2 Speed:10000 Mtu:8901} {Name:75363760a3df34a MacAddress:32:77:2f:e4:e4:37 Speed:10000 Mtu:8901} {Name:843e541ef8e2549 MacAddress:ce:38:58:67:47:1b Speed:10000 Mtu:8901} {Name:94561f7713dd00d MacAddress:6e:34:a1:fb:2c:d8 Speed:10000 Mtu:8901} {Name:ab2e8504c22308e MacAddress:aa:0e:d8:f3:82:8b Speed:10000 Mtu:8901} {Name:baa63197856687e MacAddress:5a:be:f5:77:ec:29 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:eb:21:84:29 Speed:0 Mtu:9001} {Name:br-int MacAddress:9a:a8:97:1a:3b:14 Speed:0 Mtu:8901} {Name:d117dd70dea2f9f MacAddress:52:0d:16:4f:5e:ab Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:eb:21:84:29 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:6e:6a:1a:51:7c:bb Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:6e:df:b7:e5:4d:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 00:56:43.098447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.098449    8029 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 00:56:43.098837 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.098506    8029 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 00:56:43.098837 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.098818    8029 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 00:56:43.098995 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.098846    8029 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-244.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 00:56:43.099037 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099000    8029 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 00:56:43.099037 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099008    8029 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 00:56:43.099037 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099019    8029 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:56:43.099037 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099035    8029 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 00:56:43.099150 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099122    8029 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 00:56:43.099299 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099290    8029 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 00:56:43.099414 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099404    8029 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 00:56:43.099451 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099441    8029 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 00:56:43.099480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099456    8029 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 00:56:43.099480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099470    8029 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 00:56:43.099645 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099482    8029 kubelet.go:405] "Adding apiserver pod source"
Oct 31 00:56:43.099645 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.099490    8029 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 00:56:43.101323 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.100823    8029 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 00:56:43.101323 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.100840    8029 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:56:43.106022 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.105915    8029 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 00:56:43.115676 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.115655    8029 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:56:43.118234 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.118204    8029 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:56:43.129849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.129832    8029 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 00:56:43.129923 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.129851    8029 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 00:56:43.129973 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.129963    8029 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 00:56:43.130236 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130223    8029 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 00:56:43.130278 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130268    8029 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 00:56:43.130402 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130392    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 00:56:43.130446 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130411    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 00:56:43.130446 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130421    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 00:56:43.130446 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130430    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 00:56:43.130446 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130439    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130448    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130458    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130466    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130477    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130487    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130507    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130520    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130551    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 00:56:43.130572 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.130561    8029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 00:56:43.134929 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.134916    8029 server.go:1263] "Started kubelet"
Oct 31 00:56:43.135012 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.134981    8029 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 00:56:43.135122 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.135074    8029 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 00:56:43.135180 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.135145    8029 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 00:56:43.135427 ip-10-0-140-244 systemd[1]: Started Kubernetes Kubelet.
Oct 31 00:56:43.135558 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.135548    8029 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 00:56:43.139163 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.139082    8029 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 00:56:43.141262 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141242    8029 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 00:56:43.141362 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141265    8029 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 00:56:43.141362 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141333    8029 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 00:50:28 +0000 UTC" deadline="2028-06-27 00:03:25.80133412 +0000 UTC"
Oct 31 00:56:43.141362 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141351    8029 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14519h6m42.659985452s"
Oct 31 00:56:43.141521 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141383    8029 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 00:56:43.141521 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141395    8029 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 00:56:43.141521 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.141431    8029 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 00:56:43.143120 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143096    8029 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 00:56:43.143120 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143116    8029 factory.go:55] Registering systemd factory
Oct 31 00:56:43.143120 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143125    8029 factory.go:223] Registration of the systemd container factory successfully
Oct 31 00:56:43.143831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143814    8029 factory.go:153] Registering CRI-O factory
Oct 31 00:56:43.143831 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143832    8029 factory.go:223] Registration of the crio container factory successfully
Oct 31 00:56:43.143970 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143846    8029 factory.go:103] Registering Raw factory
Oct 31 00:56:43.143970 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.143855    8029 manager.go:1199] Started watching for new ooms in manager
Oct 31 00:56:43.144189 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144173    8029 manager.go:319] Starting recovery of all containers
Oct 31 00:56:43.144757 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144715    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e925ec-3898-42da-a8c2-5a1998d17c48" volumeName="kubernetes.io/projected/02e925ec-3898-42da-a8c2-5a1998d17c48-kube-api-access-dtbdk" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144763    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144779    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/projected/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-kube-api-access-h8gkv" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144794    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba2e347b-434a-437a-bf36-85a85efd81bd" volumeName="kubernetes.io/configmap/ba2e347b-434a-437a-bf36-85a85efd81bd-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144813    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144826    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144838    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41c16bf9-670b-4e0d-b009-8898ccad69b5" volumeName="kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs" seLinuxMountContext=""
Oct 31 00:56:43.144849 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144850    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144865    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/configmap/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144878    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144890    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144902    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41c16bf9-670b-4e0d-b009-8898ccad69b5" volumeName="kubernetes.io/projected/41c16bf9-670b-4e0d-b009-8898ccad69b5-kube-api-access-v884q" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144914    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a" volumeName="kubernetes.io/configmap/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-serviceca" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144926    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144971    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e925ec-3898-42da-a8c2-5a1998d17c48" volumeName="kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144985    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c482845-4e0d-441c-aa58-44672e7f6bf3" volumeName="kubernetes.io/empty-dir/4c482845-4e0d-441c-aa58-44672e7f6bf3-data-volume" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.144996    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e925ec-3898-42da-a8c2-5a1998d17c48" volumeName="kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-tls" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145008    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="319e5395-cce3-4e4d-9c3c-111e8eb1baaa" volumeName="kubernetes.io/secret/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-cert" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145019    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145031    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145042    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e575cbe0-30bf-4286-a382-ffe481a27c91" volumeName="kubernetes.io/configmap/e575cbe0-30bf-4286-a382-ffe481a27c91-config-volume" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145054    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea64b244-e85f-494e-a824-9eba8ccfe22c" volumeName="kubernetes.io/empty-dir/ea64b244-e85f-494e-a824-9eba8ccfe22c-tmp-dir" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145064    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb321839-52ad-47ca-b4d6-b30b18970649" volumeName="kubernetes.io/configmap/fb321839-52ad-47ca-b4d6-b30b18970649-iptables-alerter-script" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145076    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145088    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90525854-b0fe-4589-8f36-df21ad285d0b" volumeName="kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145100    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145112    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee8d0cb-1686-4b84-ac7c-e07f8612d4ba" volumeName="kubernetes.io/empty-dir/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-volume-directive-shadow" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145126    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145145    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90525854-b0fe-4589-8f36-df21ad285d0b" volumeName="kubernetes.io/secret/90525854-b0fe-4589-8f36-df21ad285d0b-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145158    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba2e347b-434a-437a-bf36-85a85efd81bd" volumeName="kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145170    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee8d0cb-1686-4b84-ac7c-e07f8612d4ba" volumeName="kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145181    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp" seLinuxMountContext=""
Oct 31 00:56:43.145178 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145194    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145224    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea64b244-e85f-494e-a824-9eba8ccfe22c" volumeName="kubernetes.io/projected/ea64b244-e85f-494e-a824-9eba8ccfe22c-kube-api-access-zthpv" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145236    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed848351-036d-461e-9295-e8eb410759cc" volumeName="kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145248    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145259    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-grpc-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145271    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c482845-4e0d-441c-aa58-44672e7f6bf3" volumeName="kubernetes.io/secret/4c482845-4e0d-441c-aa58-44672e7f6bf3-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145282    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145296    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145309    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9789f7f7-7d7d-4cd1-b74e-fd960dbb0283" volumeName="kubernetes.io/secret/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-monitoring-plugin-cert" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145320    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145334    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" volumeName="kubernetes.io/projected/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-kube-api-access-k2ptv" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145345    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba2e347b-434a-437a-bf36-85a85efd81bd" volumeName="kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145356    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145368    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed848351-036d-461e-9295-e8eb410759cc" volumeName="kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145379    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145390    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145401    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145413    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4e40a1-88e5-493b-901d-668e910172cd" volumeName="kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-multus-daemon-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145424    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4e40a1-88e5-493b-901d-668e910172cd" volumeName="kubernetes.io/projected/8b4e40a1-88e5-493b-901d-668e910172cd-kube-api-access-j7kl8" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145437    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145448    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145459    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a" volumeName="kubernetes.io/projected/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-kube-api-access-4ftlv" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145471    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e925ec-3898-42da-a8c2-5a1998d17c48" volumeName="kubernetes.io/empty-dir/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-textfile" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145483    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145504    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145517    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145529    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="955e9bb3-d8e7-419d-9d8f-7c9af8d88e07" volumeName="kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-tmp" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145539    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-client-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145552    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145563    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852b2fe1-0192-404e-9c39-e16b622a3b9f" volumeName="kubernetes.io/configmap/852b2fe1-0192-404e-9c39-e16b622a3b9f-konnectivity-ca" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145574    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="955e9bb3-d8e7-419d-9d8f-7c9af8d88e07" volumeName="kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-tuned" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145586    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/empty-dir/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-tmp" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145617    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" volumeName="kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145629    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" volumeName="kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145641    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145653    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed848351-036d-461e-9295-e8eb410759cc" volumeName="kubernetes.io/projected/ed848351-036d-461e-9295-e8eb410759cc-kube-api-access-v2xst" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145664    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145676    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145687    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab0a4936-76a5-40eb-a7bd-0822ea7c8de9" volumeName="kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145699    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/projected/d6f46adb-16dc-4058-bfd1-d33fca2885a3-kube-api-access-ms9ng" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145710    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-federate-client-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145722    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="633346a3-a449-448c-8580-b87b22a05979" volumeName="kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145732    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6af5ed3a-6673-411b-87de-9ea59705bec6" volumeName="kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145745    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e575cbe0-30bf-4286-a382-ffe481a27c91" volumeName="kubernetes.io/empty-dir/e575cbe0-30bf-4286-a382-ffe481a27c91-tmp-dir" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145756    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145768    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145779    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145790    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145801    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145814    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145825    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145839    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-client-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145852    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c482845-4e0d-441c-aa58-44672e7f6bf3" volumeName="kubernetes.io/projected/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-api-access-wjv87" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145864    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e5138b9-caf5-4c38-adba-4d32a72b7da5" volumeName="kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145876    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90525854-b0fe-4589-8f36-df21ad285d0b" volumeName="kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145888    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9789f7f7-7d7d-4cd1-b74e-fd960dbb0283" volumeName="kubernetes.io/projected/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-kube-api-access-g95sl" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145899    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/empty-dir/c1ada52a-5abd-4603-922d-cb20665f8a18-audit-log" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145912    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145924    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e925ec-3898-42da-a8c2-5a1998d17c48" volumeName="kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145934    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145946    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c482845-4e0d-441c-aa58-44672e7f6bf3" volumeName="kubernetes.io/configmap/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145959    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="955e9bb3-d8e7-419d-9d8f-7c9af8d88e07" volumeName="kubernetes.io/projected/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-kube-api-access-w92g4" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145973    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba2e347b-434a-437a-bf36-85a85efd81bd" volumeName="kubernetes.io/projected/ba2e347b-434a-437a-bf36-85a85efd81bd-kube-api-access-jc7n2" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145984    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1ada52a-5abd-4603-922d-cb20665f8a18" volumeName="kubernetes.io/projected/c1ada52a-5abd-4603-922d-cb20665f8a18-kube-api-access-gzv22" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145996    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e575cbe0-30bf-4286-a382-ffe481a27c91" volumeName="kubernetes.io/secret/e575cbe0-30bf-4286-a382-ffe481a27c91-metrics-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146007    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee8d0cb-1686-4b84-ac7c-e07f8612d4ba" volumeName="kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-tls" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146018    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="319e5395-cce3-4e4d-9c3c-111e8eb1baaa" volumeName="kubernetes.io/projected/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-kube-api-access-dbp2x" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146031    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146042    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="852b2fe1-0192-404e-9c39-e16b622a3b9f" volumeName="kubernetes.io/secret/852b2fe1-0192-404e-9c39-e16b622a3b9f-agent-certs" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146058    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4e40a1-88e5-493b-901d-668e910172cd" volumeName="kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-cni-binary-copy" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146070    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9af1e2d1-ebb9-4941-8b61-dd640def4b9d" volumeName="kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146088    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe4641e5-a8d7-4cad-b10b-0989e80736ae" volumeName="kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.145985    8029 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146130    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146145    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90525854-b0fe-4589-8f36-df21ad285d0b" volumeName="kubernetes.io/projected/90525854-b0fe-4589-8f36-df21ad285d0b-kube-api-access-tf85m" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146157    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee8d0cb-1686-4b84-ac7c-e07f8612d4ba" volumeName="kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146168    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90525854-b0fe-4589-8f36-df21ad285d0b" volumeName="kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-env-overrides" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146180    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" volumeName="kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-binary-copy" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146192    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed848351-036d-461e-9295-e8eb410759cc" volumeName="kubernetes.io/configmap/ed848351-036d-461e-9295-e8eb410759cc-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146203    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee8d0cb-1686-4b84-ac7c-e07f8612d4ba" volumeName="kubernetes.io/projected/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-api-access-2gc2n" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146215    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146244    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3005fee5-7711-481a-89dc-8f191ef548cd" volumeName="kubernetes.io/projected/3005fee5-7711-481a-89dc-8f191ef548cd-kube-api-access-m8hhn" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146256    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146267    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6f46adb-16dc-4058-bfd1-d33fca2885a3" volumeName="kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146278    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb321839-52ad-47ca-b4d6-b30b18970649" volumeName="kubernetes.io/projected/fb321839-52ad-47ca-b4d6-b30b18970649-kube-api-access-g2j2l" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146289    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2db63efb-9319-4488-8589-2081d603b75e" volumeName="kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146301    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e575cbe0-30bf-4286-a382-ffe481a27c91" volumeName="kubernetes.io/projected/e575cbe0-30bf-4286-a382-ffe481a27c91-kube-api-access-t7jqp" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146314    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eee8d0cb-1686-4b84-ac7c-e07f8612d4ba" volumeName="kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-metrics-client-ca" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146326    8029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e925ec-3898-42da-a8c2-5a1998d17c48" volumeName="kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146337    8029 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 00:56:43.146334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.146344    8029 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 00:56:43.153312 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.151783    8029 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 00:56:43.163435 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.163417    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:56:43.170316 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.170297    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:56:43.173114 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.173086    8029 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 00:56:43.175044 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.175022    8029 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 00:56:43.175044 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.175044    8029 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 00:56:43.175154 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.175060    8029 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 00:56:43.175154 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:43.175093    8029 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 00:56:43.180232 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.180206    8029 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 00:56:43.190167 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.190093    8029 generic.go:356] "Generic (PLEG): container finished" podID="f6d2bb5ec637c932bd8834e531979055" containerID="a322d04abc7a727b5b1ca408105b58dd9e576ab819a85fbec971cea11ea85cb7" exitCode=0
Oct 31 00:56:43.195644 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.195616    8029 generic.go:356] "Generic (PLEG): container finished" podID="02e925ec-3898-42da-a8c2-5a1998d17c48" containerID="b46af0107e0065db33c7088103069582f682687c1b4e1b7f6f3d0503bd45b5c1" exitCode=0
Oct 31 00:56:43.218754 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.218737    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 00:56:43.219127 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.219105    8029 generic.go:356] "Generic (PLEG): container finished" podID="90525854-b0fe-4589-8f36-df21ad285d0b" containerID="dde362c46632d8a6ba801a42bdcaa8bed38d9c94ef2ec2738d5a40ef63980b28" exitCode=1
Oct 31 00:56:43.222618 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.222576    8029 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="eebf805781b95712cd1f5a6eed1eece5f8187856581b0d978c8a09ac2da331f7" exitCode=0
Oct 31 00:56:43.222618 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.222613    8029 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="b088c4dc1cd885acd45ec3a1915691771720f6fe667df64e6fd10d6f7ca5a3c5" exitCode=0
Oct 31 00:56:43.222763 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.222623    8029 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="d49d286b06c2ecef0b91a6ca50ea985cff4d93c61c66be485e1878c6a6ed3ec2" exitCode=0
Oct 31 00:56:43.222763 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.222634    8029 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="dbae7b9fb41c4f3aea8756bca412171409ebc074cf2f3e50eb80339469a303c1" exitCode=0
Oct 31 00:56:43.222763 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.222639    8029 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="3416f279a014ea3542d0d8bb56ab32448a57964b72e0048837304155e33fc558" exitCode=0
Oct 31 00:56:43.222763 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.222644    8029 generic.go:356] "Generic (PLEG): container finished" podID="9b0eff54-56e9-4ff4-9c31-17ca3325e0a6" containerID="47c7ca80308dab4ae7b9e2a7e172a7b3180ab2b886da0e49960ab023e6877b18" exitCode=0
Oct 31 00:56:43.230325 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.230304    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331" exitCode=0
Oct 31 00:56:43.260653 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.260632    8029 manager.go:324] Recovery completed
Oct 31 00:56:43.276186 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:43.276162    8029 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 31 00:56:43.286247 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286221    8029 cpu_manager.go:226] "Starting" policy="none"
Oct 31 00:56:43.286247 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286238    8029 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 00:56:43.286377 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286257    8029 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 00:56:43.286411 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286396    8029 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 00:56:43.286445 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286411    8029 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 00:56:43.286445 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286434    8029 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 00:56:43.286445 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286441    8029 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 00:56:43.286524 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286448    8029 policy_none.go:50] "Start"
Oct 31 00:56:43.286524 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286460    8029 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 00:56:43.286524 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.286473    8029 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 00:56:43.288130 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.288117    8029 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 00:56:43.288170 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.288130    8029 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 00:56:43.288170 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.288136    8029 policy_none.go:44] "Start"
Oct 31 00:56:43.293324 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.292830    8029 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 00:56:43.298646 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.298631    8029 manager.go:341] "Starting Device Plugin manager"
Oct 31 00:56:43.298711 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:43.298657    8029 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 00:56:43.298711 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.298666    8029 server.go:87] "Starting device plugin registration server"
Oct 31 00:56:43.298981 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.298969    8029 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 00:56:43.299013 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.298984    8029 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 00:56:43.299119 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.299095    8029 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 00:56:43.299262 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.299210    8029 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 00:56:43.299380 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.299282    8029 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 00:56:43.299380 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.299290    8029 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 00:56:43.299380 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.299323    8029 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T00:56:43.299257493Z","UUID":"ea947096-2c90-41ee-aa42-51b348e96ced","Handler":null,"Name":"","Endpoint":""}
Oct 31 00:56:43.300521 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.300505    8029 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 00:56:43.300620 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.300540    8029 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 00:56:43.301064 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:43.301045    8029 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 00:56:43.399345 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.399323    8029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 00:56:43.401161 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.401146    8029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 00:56:43.401210 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.401175    8029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 00:56:43.401210 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.401186    8029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-244.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 00:56:43.401291 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.401229    8029 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.422469 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.422448    8029 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.422534 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.422503    8029 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.477027 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.476953    8029 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"]
Oct 31 00:56:43.477585 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477546    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"f6d2bb5ec637c932bd8834e531979055","Type":"ContainerStarted","Data":"e69eebfaa35701d8f33f2272753b64f183a3d5f61ca713311f04549de4d999ee"}
Oct 31 00:56:43.477652 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477595    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"f6d2bb5ec637c932bd8834e531979055","Type":"ContainerDied","Data":"a322d04abc7a727b5b1ca408105b58dd9e576ab819a85fbec971cea11ea85cb7"}
Oct 31 00:56:43.477652 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477636    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"f6d2bb5ec637c932bd8834e531979055","Type":"ContainerStarted","Data":"92386904780ec6910838a5c53a1a2d1e3094a1a6b664288611d6e9152c1ba08b"}
Oct 31 00:56:43.477709 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477684    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"751e5bfb25820222e24aac2c7122de2e","Type":"ContainerStarted","Data":"1ef24b55310bedc8525b6bdc27b260f7cb043d17b91f69a65df4d530e339d664"}
Oct 31 00:56:43.477709 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477576    8029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.477709 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477703    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal" event={"ID":"751e5bfb25820222e24aac2c7122de2e","Type":"ContainerStarted","Data":"f9f359604c595a29430034cd22e617183e85063541b9b041d3ddf9e04d056e23"}
Oct 31 00:56:43.477811 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.477786    8029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.512134 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.512115    8029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:56:43.512204 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:43.512151    8029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.513698 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.513684    8029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:56:43.513752 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:43.513735    8029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.552959 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.552932    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.553046 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.553010    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.553081 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.553053    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/751e5bfb25820222e24aac2c7122de2e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"751e5bfb25820222e24aac2c7122de2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.653530 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.653491    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.653530 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.653531    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/751e5bfb25820222e24aac2c7122de2e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"751e5bfb25820222e24aac2c7122de2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.653681 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.653550    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.653681 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.653590    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.653681 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.653624    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d2bb5ec637c932bd8834e531979055-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"f6d2bb5ec637c932bd8834e531979055\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:43.653681 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:43.653625    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/751e5bfb25820222e24aac2c7122de2e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" (UID: \"751e5bfb25820222e24aac2c7122de2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:44.100031 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.100001    8029 apiserver.go:52] "Watching apiserver"
Oct 31 00:56:44.115985 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.115951    8029 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 00:56:44.116869 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.116847    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z9b9p","openshift-monitoring/prometheus-k8s-0","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal","openshift-dns/dns-default-p9js6","openshift-monitoring/telemeter-client-68b5c76845-8wplb","openshift-dns/node-resolver-ddx8v","openshift-console/console-5f44b4d54f-scl9x","openshift-image-registry/node-ca-hjrz2","openshift-monitoring/metrics-server-6db4c97dcc-87jqk","kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-tmhd2","openshift-insights/insights-runtime-extractor-p2bzn","openshift-multus/network-metrics-daemon-7r985","openshift-ingress-canary/ingress-canary-r9zb2","kube-system/konnectivity-agent-m9m29","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk","kube-system/global-pull-secret-syncer-6tsqv","openshift-monitoring/monitoring-plugin-77f4f76594-sddw2","openshift-monitoring/thanos-querier-5c7fd74876-fxzt7","openshift-multus/multus-2m96p","openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l","openshift-ovn-kubernetes/ovnkube-node-2z92c","openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7","openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd","openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5","openshift-multus/multus-additional-cni-plugins-d6xbt","openshift-network-diagnostics/network-check-target-rs9gd","openshift-network-operator/iptables-alerter-vh94r","openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:56:44.123506 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.123485    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jrg45\""
Oct 31 00:56:44.123653 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.123532    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 00:56:44.123976 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.123948    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89pfs\""
Oct 31 00:56:44.124430 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.124411    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xcm4p\""
Oct 31 00:56:44.124525 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.124473    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 00:56:44.124571 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.124559    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-xb7hg\""
Oct 31 00:56:44.124648 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.124632    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 00:56:44.127871 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.127850    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 00:56:44.127984 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.127895    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xxmpd\""
Oct 31 00:56:44.127984 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.127930    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.128115 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.127989    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.128342 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128327    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.129071 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128536    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9dpj5\""
Oct 31 00:56:44.129071 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128543    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.129071 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128563    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 00:56:44.129071 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128584    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.129071 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128590    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.129359 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129294    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 00:56:44.129359 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129301    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 00:56:44.129359 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129316    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.129359 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129316    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.129359 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129324    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-89rf9\""
Oct 31 00:56:44.129359 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129355    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.129640 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129620    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4j5v2\""
Oct 31 00:56:44.129752 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.128540    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 00:56:44.129752 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129741    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 00:56:44.129867 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129809    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 00:56:44.129867 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.129824    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.130592 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.130577    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 00:56:44.131047 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.131034    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 00:56:44.131047 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.131035    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 00:56:44.131182 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.131098    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gczx4\""
Oct 31 00:56:44.131410 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.131393    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j9qsc\""
Oct 31 00:56:44.131656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.131636    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dlpqk\""
Oct 31 00:56:44.131770 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.131685    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m2wnh\""
Oct 31 00:56:44.132128 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.132112    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.132300 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.132282    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.132349 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.132290    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 00:56:44.133231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133191    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.133231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133209    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.133231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133225    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 00:56:44.133436 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133277    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.133436 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133299    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.133436 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133320    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.133436 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133329    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-z2mss\""
Oct 31 00:56:44.133436 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133360    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 00:56:44.133436 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133383    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.133901 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133888    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-58wnq\""
Oct 31 00:56:44.133963 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.133941    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 00:56:44.142446 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.142428    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 00:56:44.143173 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143153    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.143251 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143222    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.143379 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143364    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 00:56:44.143470 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143427    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.143470 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143437    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.143470 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143463    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 00:56:44.143577 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143523    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 00:56:44.143577 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143542    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 00:56:44.143938 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143913    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p5vm9\""
Oct 31 00:56:44.143938 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143916    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f7djw\""
Oct 31 00:56:44.144038 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143941    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-55222\""
Oct 31 00:56:44.144038 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143940    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 00:56:44.144038 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.143944    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 00:56:44.144211 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.144189    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-72bs2\""
Oct 31 00:56:44.144487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.144473    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 00:56:44.144487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.144481    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 31 00:56:44.144630 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.144580    8029 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 00:56:44.144681 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.144630    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 00:56:44.144681 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.144640    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 00:56:44.155452 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155430    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-client-certs\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.155518 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155463    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.155518 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155481    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.155518 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155497    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc7n2\" (UniqueName: \"kubernetes.io/projected/ba2e347b-434a-437a-bf36-85a85efd81bd-kube-api-access-jc7n2\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.155652 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155637    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-k8s-cni-cncf-io\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.155686 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155667    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-etc-kubernetes\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.155724 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155684    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-netns\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.155724 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155707    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ftlv\" (UniqueName: \"kubernetes.io/projected/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-kube-api-access-4ftlv\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.155796 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155743    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.155833 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155798    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.155863 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155831    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-mqpr5\" (UID: \"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:44.155863 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155849    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.155951 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155865    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-multus-daemon-config\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.155951 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155883    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-log-socket\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.155951 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155898    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90525854-b0fe-4589-8f36-df21ad285d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.155951 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155914    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-sys\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.156141 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155936    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-host\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.156141 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.155994    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.156141 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156032    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.156141 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156048    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab0a4936-76a5-40eb-a7bd-0822ea7c8de9-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-mqpr5\" (UID: \"ab0a4936-76a5-40eb-a7bd-0822ea7c8de9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:44.156141 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156072    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156142    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/633346a3-a449-448c-8580-b87b22a05979-original-pull-secret\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156163    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c482845-4e0d-441c-aa58-44672e7f6bf3-data-volume\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156170    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90525854-b0fe-4589-8f36-df21ad285d0b-ovn-node-metrics-cert\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156201    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c482845-4e0d-441c-aa58-44672e7f6bf3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156214    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156230    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156256    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c482845-4e0d-441c-aa58-44672e7f6bf3-data-volume\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156259    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-client-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.156322 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156305    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156333    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-var-lib-kubelet\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156353    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-tls\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156370    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156363    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c482845-4e0d-441c-aa58-44672e7f6bf3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156393    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7jqp\" (UniqueName: \"kubernetes.io/projected/e575cbe0-30bf-4286-a382-ffe481a27c91-kube-api-access-t7jqp\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156422    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed848351-036d-461e-9295-e8eb410759cc-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156445    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-metrics-client-ca\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156539    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8gkv\" (UniqueName: \"kubernetes.io/projected/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-kube-api-access-h8gkv\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156567    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156588    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-metrics-client-ca\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156591    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/852b2fe1-0192-404e-9c39-e16b622a3b9f-agent-certs\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:56:44.156656 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156640    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-var-lib-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156664    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-registration-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156666    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed848351-036d-461e-9295-e8eb410759cc-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156687    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-sys-fs\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156746    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-host\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156778    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95sl\" (UniqueName: \"kubernetes.io/projected/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-kube-api-access-g95sl\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156794    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156810    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e575cbe0-30bf-4286-a382-ffe481a27c91-tmp-dir\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156835    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156853    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e575cbe0-30bf-4286-a382-ffe481a27c91-tmp-dir\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156859    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156866    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/852b2fe1-0192-404e-9c39-e16b622a3b9f-agent-certs\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156890    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156908    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156924    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-metrics-client-ca\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156946    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-etc-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156966    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c482845-4e0d-441c-aa58-44672e7f6bf3-crio-socket\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156982    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156993    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.156997    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157013    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157031    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157088    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-device-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157096    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157108    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-socket-dir-parent\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157138    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb321839-52ad-47ca-b4d6-b30b18970649-iptables-alerter-script\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157161    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gc2n\" (UniqueName: \"kubernetes.io/projected/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-api-access-2gc2n\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.157217 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157177    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157242    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-metrics-client-ca\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157289    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-kubelet\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157311    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb321839-52ad-47ca-b4d6-b30b18970649-iptables-alerter-script\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157324    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-systemd-units\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157348    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-systemd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157372    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cnibin\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157402    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-serving-certs-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157427    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-modprobe-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157442    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-systemd\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157457    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea64b244-e85f-494e-a824-9eba8ccfe22c-tmp-dir\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157505    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-serviceca\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157529    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157556    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2xst\" (UniqueName: \"kubernetes.io/projected/ed848351-036d-461e-9295-e8eb410759cc-kube-api-access-v2xst\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157563    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea64b244-e85f-494e-a824-9eba8ccfe22c-tmp-dir\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157580    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-federate-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157650    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157682    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157709    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-system-cni-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157753    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-metrics-server-audit-profiles\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157771    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-serviceca\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157849    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157887    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-hostroot\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157913    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-grpc-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.157981    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158005    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158026    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-dbus\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158044    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-tls\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158059    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7kl8\" (UniqueName: \"kubernetes.io/projected/8b4e40a1-88e5-493b-901d-668e910172cd-kube-api-access-j7kl8\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158076    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158102    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158107    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158138    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.158231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158151    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w92g4\" (UniqueName: \"kubernetes.io/projected/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-kube-api-access-w92g4\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158273    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158312    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjv87\" (UniqueName: \"kubernetes.io/projected/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-api-access-wjv87\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158390    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-accelerators-collector-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158423    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c1ada52a-5abd-4603-922d-cb20665f8a18-audit-log\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158450    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158474    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-multus\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158499    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysconfig\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158512    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c1ada52a-5abd-4603-922d-cb20665f8a18-audit-log\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158527    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-system-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158554    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-run\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158583    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-script-lib\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158639    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-cert\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158718    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158750    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158780    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf85m\" (UniqueName: \"kubernetes.io/projected/90525854-b0fe-4589-8f36-df21ad285d0b-kube-api-access-tf85m\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158807    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbp2x\" (UniqueName: \"kubernetes.io/projected/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-kube-api-access-dbp2x\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158826    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-script-lib\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158834    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e575cbe0-30bf-4286-a382-ffe481a27c91-config-volume\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158828    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-cert\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158877    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158895    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-slash\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158903    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158910    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb321839-52ad-47ca-b4d6-b30b18970649-host-slash\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.158949    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159000    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e575cbe0-30bf-4286-a382-ffe481a27c91-config-volume\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159052    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159085    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159113    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159141    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159166    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-node-log\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159197    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159221    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159247    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159278    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-netns\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159335    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159354    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159384    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-sys\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159411    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-root\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159464    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzv22\" (UniqueName: \"kubernetes.io/projected/c1ada52a-5abd-4603-922d-cb20665f8a18-kube-api-access-gzv22\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159486    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159505    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159521    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-conf-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159546    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-ovn\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.159547 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159565    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-netd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159593    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2ptv\" (UniqueName: \"kubernetes.io/projected/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-kube-api-access-k2ptv\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159651    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159675    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e575cbe0-30bf-4286-a382-ffe481a27c91-metrics-tls\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159717    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159764    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-kubelet-config\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159772    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159807    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159842    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159876    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e575cbe0-30bf-4286-a382-ffe481a27c91-metrics-tls\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159888    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zthpv\" (UniqueName: \"kubernetes.io/projected/ea64b244-e85f-494e-a824-9eba8ccfe22c-kube-api-access-zthpv\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159890    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.159968    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-cni-binary-copy\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160047    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c16bf9-670b-4e0d-b009-8898ccad69b5-metrics-certs\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160059    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-bin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160078    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160097    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160113    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160128    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-lib-modules\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160149    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-env-overrides\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160187    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-metrics-client-ca\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160189    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-cni-binary-copy\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160231    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160245    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160271    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2e347b-434a-437a-bf36-85a85efd81bd-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160288    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8hhn\" (UniqueName: \"kubernetes.io/projected/3005fee5-7711-481a-89dc-8f191ef548cd-kube-api-access-m8hhn\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160308    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-env-overrides\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160360    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea64b244-e85f-494e-a824-9eba8ccfe22c-hosts-file\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160365    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-metrics-client-ca\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160384    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160421    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160468    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-tmp\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160480    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160502    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160532    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-tmp\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160537    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160487    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2e347b-434a-437a-bf36-85a85efd81bd-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160580    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/852b2fe1-0192-404e-9c39-e16b622a3b9f-konnectivity-ca\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160640    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160676    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v884q\" (UniqueName: \"kubernetes.io/projected/41c16bf9-670b-4e0d-b009-8898ccad69b5-kube-api-access-v884q\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160681    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160704    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160731    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-wtmp\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160757    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160795    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-cnibin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160786    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/852b2fe1-0192-404e-9c39-e16b622a3b9f-konnectivity-ca\") pod \"konnectivity-agent-m9m29\" (UID: \"852b2fe1-0192-404e-9c39-e16b622a3b9f\") " pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160845    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-multus-certs\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160887    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160925    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160954    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-kubelet\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160976    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-conf\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.160991    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-config\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161002    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161008    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161004    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161045    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161113    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161136    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-bin\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161152    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161173    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161181    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161212    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90525854-b0fe-4589-8f36-df21ad285d0b-ovnkube-config\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161217    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:44.161235    8029 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:44.161237 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161244    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9ng\" (UniqueName: \"kubernetes.io/projected/d6f46adb-16dc-4058-bfd1-d33fca2885a3-kube-api-access-ms9ng\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:44.161326    8029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle podName:fe4641e5-a8d7-4cad-b10b-0989e80736ae nodeName:}" failed. No retries permitted until 2026-10-31 00:56:44.661305485 +0000 UTC m=+1.648417439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle") pod "console-6db8b47f85-pkjfx" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161389    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-tmp\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161438    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-tmp\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161464    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-tuned\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161497    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161525    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161552    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161563    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-tuned\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161580    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-socket-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161623    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-kubernetes\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161660    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161695    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-textfile\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161723    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161730    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtbdk\" (UniqueName: \"kubernetes.io/projected/02e925ec-3898-42da-a8c2-5a1998d17c48-kube-api-access-dtbdk\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161770    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161776    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-textfile\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161802    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2j2l\" (UniqueName: \"kubernetes.io/projected/fb321839-52ad-47ca-b4d6-b30b18970649-kube-api-access-g2j2l\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.163136 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.161873    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.174259 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.173976    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8gkv\" (UniqueName: \"kubernetes.io/projected/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-kube-api-access-h8gkv\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.174259 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.173995    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ftlv\" (UniqueName: \"kubernetes.io/projected/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-kube-api-access-4ftlv\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.174259 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174094    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95sl\" (UniqueName: \"kubernetes.io/projected/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-kube-api-access-g95sl\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:44.174259 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174206    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gc2n\" (UniqueName: \"kubernetes.io/projected/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-api-access-2gc2n\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.174457 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174332    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7jqp\" (UniqueName: \"kubernetes.io/projected/e575cbe0-30bf-4286-a382-ffe481a27c91-kube-api-access-t7jqp\") pod \"dns-default-p9js6\" (UID: \"e575cbe0-30bf-4286-a382-ffe481a27c91\") " pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:44.174457 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174342    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7kl8\" (UniqueName: \"kubernetes.io/projected/8b4e40a1-88e5-493b-901d-668e910172cd-kube-api-access-j7kl8\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.174457 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174349    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc7n2\" (UniqueName: \"kubernetes.io/projected/ba2e347b-434a-437a-bf36-85a85efd81bd-kube-api-access-jc7n2\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.174544 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174469    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w92g4\" (UniqueName: \"kubernetes.io/projected/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-kube-api-access-w92g4\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.174573 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174558    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2xst\" (UniqueName: \"kubernetes.io/projected/ed848351-036d-461e-9295-e8eb410759cc-kube-api-access-v2xst\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.174888 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174872    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsdf\" (UniqueName: \"kubernetes.io/projected/5e5138b9-caf5-4c38-adba-4d32a72b7da5-kube-api-access-4wsdf\") pod \"network-check-target-rs9gd\" (UID: \"5e5138b9-caf5-4c38-adba-4d32a72b7da5\") " pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:44.174932 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.174916    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzv22\" (UniqueName: \"kubernetes.io/projected/c1ada52a-5abd-4603-922d-cb20665f8a18-kube-api-access-gzv22\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.176029 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.176009    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf85m\" (UniqueName: \"kubernetes.io/projected/90525854-b0fe-4589-8f36-df21ad285d0b-kube-api-access-tf85m\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.176147 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.176110    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.176213 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.176198    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zthpv\" (UniqueName: \"kubernetes.io/projected/ea64b244-e85f-494e-a824-9eba8ccfe22c-kube-api-access-zthpv\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.176396 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.176376    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v884q\" (UniqueName: \"kubernetes.io/projected/41c16bf9-670b-4e0d-b009-8898ccad69b5-kube-api-access-v884q\") pod \"network-metrics-daemon-7r985\" (UID: \"41c16bf9-670b-4e0d-b009-8898ccad69b5\") " pod="openshift-multus/network-metrics-daemon-7r985"
Oct 31 00:56:44.176593 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.176580    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjv87\" (UniqueName: \"kubernetes.io/projected/4c482845-4e0d-441c-aa58-44672e7f6bf3-kube-api-access-wjv87\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.176905 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.176891    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8hhn\" (UniqueName: \"kubernetes.io/projected/3005fee5-7711-481a-89dc-8f191ef548cd-kube-api-access-m8hhn\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.177493 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.177477    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2ptv\" (UniqueName: \"kubernetes.io/projected/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-kube-api-access-k2ptv\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.179986 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.179967    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbp2x\" (UniqueName: \"kubernetes.io/projected/319e5395-cce3-4e4d-9c3c-111e8eb1baaa-kube-api-access-dbp2x\") pod \"ingress-canary-r9zb2\" (UID: \"319e5395-cce3-4e4d-9c3c-111e8eb1baaa\") " pod="openshift-ingress-canary/ingress-canary-r9zb2"
Oct 31 00:56:44.180192 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.180174    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.180246 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.180237    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtbdk\" (UniqueName: \"kubernetes.io/projected/02e925ec-3898-42da-a8c2-5a1998d17c48-kube-api-access-dtbdk\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.181158 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.181140    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.181453 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.181436    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9ng\" (UniqueName: \"kubernetes.io/projected/d6f46adb-16dc-4058-bfd1-d33fca2885a3-kube-api-access-ms9ng\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.181799 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.181781    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2j2l\" (UniqueName: \"kubernetes.io/projected/fb321839-52ad-47ca-b4d6-b30b18970649-kube-api-access-g2j2l\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.225016 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.224991    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 31 00:56:44.225156 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225138    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 00:56:44.225208 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225170    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 00:56:44.225239 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225211    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 00:56:44.225239 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225231    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 00:56:44.225438 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225427    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 00:56:44.225657 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225646    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 31 00:56:44.225752 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225736    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 31 00:56:44.225795 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225757    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 31 00:56:44.225858 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.225846    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 00:56:44.226120 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.226093    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed848351-036d-461e-9295-e8eb410759cc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-dzqr7\" (UID: \"ed848351-036d-461e-9295-e8eb410759cc\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-dzqr7"
Oct 31 00:56:44.226208 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.226132    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 00:56:44.226208 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.226145    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b4e40a1-88e5-493b-901d-668e910172cd-multus-daemon-config\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.226543 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.226521    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 00:56:44.226616 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.226552    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.227134 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.227116    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.227242 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.227224    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 00:56:44.227321 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.227297    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 31 00:56:44.227475 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.227458    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.227655 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.227640    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-serving-certs-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.228088 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.228072    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-metrics-server-audit-profiles\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.228347 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.228330    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.228854 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.228839    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-accelerators-collector-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.229992 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.229973    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.230033 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.229956    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.231297 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.231283    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.231382 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.231367    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.231818 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.231803    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1ada52a-5abd-4603-922d-cb20665f8a18-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.232790 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232777    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qgqml\""
Oct 31 00:56:44.232853 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232836    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 31 00:56:44.232853 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232846    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 31 00:56:44.232937 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232861    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 31 00:56:44.232979 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232931    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 00:56:44.232979 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232944    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 31 00:56:44.232979 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232936    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 31 00:56:44.233080 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.232990    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d8z8j\""
Oct 31 00:56:44.233080 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.233022    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 31 00:56:44.233080 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.233036    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 31 00:56:44.233871 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.233856    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 00:56:44.233871 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.233864    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 00:56:44.233964 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.233894    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 00:56:44.234014 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234001    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 00:56:44.234059 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234029    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 00:56:44.234059 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234041    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 00:56:44.234177 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234164    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-w8f42\""
Oct 31 00:56:44.234231 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234191    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 31 00:56:44.234593 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234579    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 00:56:44.234593 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.234583    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w6v48\""
Oct 31 00:56:44.235045 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235032    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 31 00:56:44.235082 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235035    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 31 00:56:44.235082 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235078    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 31 00:56:44.235228 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235214    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 31 00:56:44.235457 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235441    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-an58pn2d2p7d5\""
Oct 31 00:56:44.235543 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235526    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vtd8r\""
Oct 31 00:56:44.235640 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235549    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 00:56:44.235686 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235640    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2nprh\""
Oct 31 00:56:44.235686 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235665    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-57q8fd87sthu0\""
Oct 31 00:56:44.235686 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235640    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 31 00:56:44.235841 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235705    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9qklh\""
Oct 31 00:56:44.235841 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235727    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 00:56:44.235841 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235794    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3u451jijcguql\""
Oct 31 00:56:44.235841 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235832    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7m7mt\""
Oct 31 00:56:44.236004 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235929    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 31 00:56:44.236004 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.235929    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.236074 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236008    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.236074 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236010    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-client-certs\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.236151 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236132    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.236661 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236570    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.236661 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236582    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-client-ca-bundle\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.236661 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236588    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.236856 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.236807    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-tls\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.237738 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.237719    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.237851 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.237816    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") pod \"console-5f44b4d54f-scl9x\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") " pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:44.238089 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.238072    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-federate-client-tls\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.238497 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.238468    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c1ada52a-5abd-4603-922d-cb20665f8a18-secret-metrics-server-tls\") pod \"metrics-server-6db4c97dcc-87jqk\" (UID: \"c1ada52a-5abd-4603-922d-cb20665f8a18\") " pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:44.238794 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.238747    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.239174 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.239154    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 00:56:44.240890 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.240823    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.240890 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.240836    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-thanos-querier-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.241110 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.240865    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.241110 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241042    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.241110 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241055    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9af1e2d1-ebb9-4941-8b61-dd640def4b9d-secret-grpc-tls\") pod \"thanos-querier-5c7fd74876-fxzt7\" (UID: \"9af1e2d1-ebb9-4941-8b61-dd640def4b9d\") " pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:44.241110 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241034    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.241300 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241117    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 00:56:44.241300 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241196    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9789f7f7-7d7d-4cd1-b74e-fd960dbb0283-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-sddw2\" (UID: \"9789f7f7-7d7d-4cd1-b74e-fd960dbb0283\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:44.241300 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241232    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.241300 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241230    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2e347b-434a-437a-bf36-85a85efd81bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rxckd\" (UID: \"ba2e347b-434a-437a-bf36-85a85efd81bd\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rxckd"
Oct 31 00:56:44.241468 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241354    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f46adb-16dc-4058-bfd1-d33fca2885a3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.241468 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241418    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.242329 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241483    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.242329 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241553    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eee8d0cb-1686-4b84-ac7c-e07f8612d4ba-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-79z4l\" (UID: \"eee8d0cb-1686-4b84-ac7c-e07f8612d4ba\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-79z4l"
Oct 31 00:56:44.242329 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.241563    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.242329 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.242141    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 31 00:56:44.242329 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.242253    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.242546 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.242338    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.243420 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.243110    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.246858 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.246842    8029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:44.246963 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.246949    8029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:44.250184 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.250166    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:44.250961 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.250944    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6f46adb-16dc-4058-bfd1-d33fca2885a3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68b5c76845-8wplb\" (UID: \"d6f46adb-16dc-4058-bfd1-d33fca2885a3\") " pod="openshift-monitoring/telemeter-client-68b5c76845-8wplb"
Oct 31 00:56:44.254653 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.254638    8029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:56:44.254713 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:44.254673    8029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:44.260670 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.260656    8029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 00:56:44.260734 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:44.260687    8029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-244.us-east-2.compute.internal"
Oct 31 00:56:44.262721 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262705    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-bin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262794 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262728    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-lib-modules\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.262794 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262746    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea64b244-e85f-494e-a824-9eba8ccfe22c-hosts-file\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.262794 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262762    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.262794 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262780    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-wtmp\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262825    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262839    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ea64b244-e85f-494e-a824-9eba8ccfe22c-hosts-file\") pod \"node-resolver-ddx8v\" (UID: \"ea64b244-e85f-494e-a824-9eba8ccfe22c\") " pod="openshift-dns/node-resolver-ddx8v"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262839    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-bin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262843    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-cnibin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262896    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-cnibin\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262897    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-lib-modules\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262905    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-node-exporter-wtmp\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262924    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-multus-certs\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262895    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-multus-certs\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262961    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-kubelet\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262987    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-conf\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.262994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.262998    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-kubelet\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263015    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-bin\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263054    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263113    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-bin\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263128    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-conf\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263130    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-socket-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263156    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-kubernetes\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263166    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263182    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263216    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-k8s-cni-cncf-io\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263234    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263244    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-etc-kubernetes\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263252    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-kubernetes\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263255    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-socket-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263260    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-netns\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263278    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-k8s-cni-cncf-io\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263285    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-etc-kubernetes\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263317    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-netns\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263345    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-log-socket\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263368    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-sys\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263389    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-host\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263396    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-log-socket\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263415    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-var-lib-kubelet\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263436    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-sys\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.263447 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263454    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a-host\") pod \"node-ca-hjrz2\" (UID: \"f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a\") " pod="openshift-image-registry/node-ca-hjrz2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263469    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-var-lib-kubelet\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263475    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-var-lib-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263494    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-registration-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263518    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-sys-fs\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263523    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-var-lib-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263540    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-host\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263549    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-registration-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263568    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-etc-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263587    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c482845-4e0d-441c-aa58-44672e7f6bf3-crio-socket\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263570    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-sys-fs\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263626    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-device-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263629    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-etc-openvswitch\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263671    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-socket-dir-parent\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263676    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-device-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263685    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c482845-4e0d-441c-aa58-44672e7f6bf3-crio-socket\") pod \"insights-runtime-extractor-p2bzn\" (UID: \"4c482845-4e0d-441c-aa58-44672e7f6bf3\") " pod="openshift-insights/insights-runtime-extractor-p2bzn"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263692    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-kubelet\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263708    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-host\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263719    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-systemd-units\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263714    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-kubelet\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263733    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-socket-dir-parent\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263750    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-systemd-units\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263769    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-systemd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263794    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cnibin\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263810    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-systemd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263819    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-modprobe-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263830    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-cnibin\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263840    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-systemd\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263862    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263882    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263886    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-systemd\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263899    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-modprobe-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263906    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-system-cni-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263929    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-hostroot\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263935    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263941    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263943    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-dbus\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263983    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-system-cni-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263987    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-hostroot\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.263989    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264018    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264027    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-dbus\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264034    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264056    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-multus\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264081    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysconfig\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264092    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b0eff54-56e9-4ff4-9c31-17ca3325e0a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6xbt\" (UID: \"9b0eff54-56e9-4ff4-9c31-17ca3325e0a6\") " pod="openshift-multus/multus-additional-cni-plugins-d6xbt"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264102    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-system-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264107    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3005fee5-7711-481a-89dc-8f191ef548cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsbjk\" (UID: \"3005fee5-7711-481a-89dc-8f191ef548cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsbjk"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264115    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysctl-d\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264108    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-var-lib-cni-multus\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264123    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-run\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264145    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-etc-sysconfig\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264149    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-slash\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264156    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-system-cni-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264181    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb321839-52ad-47ca-b4d6-b30b18970649-host-slash\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264187    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-slash\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264199    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-node-log\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264202    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-run\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264214    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-netns\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264229    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-sys\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264240    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb321839-52ad-47ca-b4d6-b30b18970649-host-slash\") pod \"iptables-alerter-vh94r\" (UID: \"fb321839-52ad-47ca-b4d6-b30b18970649\") " pod="openshift-network-operator/iptables-alerter-vh94r"
Oct 31 00:56:44.264216 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264242    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-root\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264262    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-host-run-netns\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264270    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/955e9bb3-d8e7-419d-9d8f-7c9af8d88e07-sys\") pod \"tuned-tmhd2\" (UID: \"955e9bb3-d8e7-419d-9d8f-7c9af8d88e07\") " pod="openshift-cluster-node-tuning-operator/tuned-tmhd2"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264264    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-conf-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264281    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b4e40a1-88e5-493b-901d-668e910172cd-multus-conf-dir\") pod \"multus-2m96p\" (UID: \"8b4e40a1-88e5-493b-901d-668e910172cd\") " pod="openshift-multus/multus-2m96p"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264290    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-node-log\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264292    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-ovn\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264305    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02e925ec-3898-42da-a8c2-5a1998d17c48-root\") pod \"node-exporter-z9b9p\" (UID: \"02e925ec-3898-42da-a8c2-5a1998d17c48\") " pod="openshift-monitoring/node-exporter-z9b9p"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264309    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-run-ovn\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264319    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-netd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264337    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-kubelet-config\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264394    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/633346a3-a449-448c-8580-b87b22a05979-kubelet-config\") pod \"global-pull-secret-syncer-6tsqv\" (UID: \"633346a3-a449-448c-8580-b87b22a05979\") " pod="kube-system/global-pull-secret-syncer-6tsqv"
Oct 31 00:56:44.265885 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.264400    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90525854-b0fe-4589-8f36-df21ad285d0b-host-cni-netd\") pod \"ovnkube-node-2z92c\" (UID: \"90525854-b0fe-4589-8f36-df21ad285d0b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:44.425292 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.425273    8029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 00:56:44.667326 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:44.667282    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:44.667455 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:44.667422    8029 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:44.667490 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:44.667484    8029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle podName:fe4641e5-a8d7-4cad-b10b-0989e80736ae nodeName:}" failed. No retries permitted until 2026-10-31 00:56:45.667466964 +0000 UTC m=+2.654578918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle") pod "console-6db8b47f85-pkjfx" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:45.255658 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:45.255565    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rs9gd" event={"ID":"5e5138b9-caf5-4c38-adba-4d32a72b7da5","Type":"ContainerStarted","Data":"9b98bd72ccab6da90d514da4f26d64ae47b057dfcefee363036a6143fa13006e"}
Oct 31 00:56:45.640616 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:45.640580    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:56:45.679184 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:45.679154    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:45.679345 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:45.679257    8029 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:45.679345 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:45.679336    8029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle podName:fe4641e5-a8d7-4cad-b10b-0989e80736ae nodeName:}" failed. No retries permitted until 2026-10-31 00:56:47.679319333 +0000 UTC m=+4.666431304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle") pod "console-6db8b47f85-pkjfx" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:46.857539 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:46.857507    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p9js6"
Oct 31 00:56:47.700653 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:47.700618    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:47.700758 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:47.700670    8029 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:47.700758 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:47.700730    8029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle podName:fe4641e5-a8d7-4cad-b10b-0989e80736ae nodeName:}" failed. No retries permitted until 2026-10-31 00:56:51.700717999 +0000 UTC m=+8.687829953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle") pod "console-6db8b47f85-pkjfx" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:48.271283 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.271252    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"}
Oct 31 00:56:48.271759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.271316    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"}
Oct 31 00:56:48.271759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.271343    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"}
Oct 31 00:56:48.271759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.271364    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"}
Oct 31 00:56:48.271759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.271388    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"}
Oct 31 00:56:48.271759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.271408    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerStarted","Data":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"}
Oct 31 00:56:48.426186 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:48.426159    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:49.062193 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:49.062162    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:56:49.261414 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:49.261377    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c7fd74876-fxzt7"
Oct 31 00:56:49.441457 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:49.441423    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 00:56:49.496838 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:49.496805    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:56:49.497025 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:49.497004    8029 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 00:56:49.786198 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:49.786111    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:49.848582 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:49.848557    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:50.279666 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:50.279627    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:50.295318 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:50.295296    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:56:51.744058 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:51.744021    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:51.744456 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:51.744114    8029 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:51.744456 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:56:51.744167    8029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle podName:fe4641e5-a8d7-4cad-b10b-0989e80736ae nodeName:}" failed. No retries permitted until 2026-10-31 00:56:59.744155633 +0000 UTC m=+16.731267588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle") pod "console-6db8b47f85-pkjfx" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 31 00:56:53.105330 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:53.105290    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f44b4d54f-scl9x"]
Oct 31 00:56:53.704731 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:53.704698    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-mqpr5"
Oct 31 00:56:54.018041 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:54.017959    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:54.022468 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:54.022444    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-sddw2"
Oct 31 00:56:54.734450 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:54.734415    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:54.738074 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:54.738046    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:55.595309 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.595270    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6599b76c78-26thf"]
Oct 31 00:56:55.665381 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.665349    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6599b76c78-26thf"]
Oct 31 00:56:55.665544 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.665461    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.674234 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.674215    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 00:56:55.785120 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785091    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-serving-cert\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785130    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c0051e9-8149-4f06-a8c6-147bd62d8436-tmp\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785160    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqq69\" (UniqueName: \"kubernetes.io/projected/9c0051e9-8149-4f06-a8c6-147bd62d8436-kube-api-access-mqq69\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785283    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-config\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785340    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-oauth-config\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785377    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-trusted-ca-bundle\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785480 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785471    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-service-ca\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.785705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.785492    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-oauth-serving-cert\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886156 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886128    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-serving-cert\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886160    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c0051e9-8149-4f06-a8c6-147bd62d8436-tmp\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886178    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqq69\" (UniqueName: \"kubernetes.io/projected/9c0051e9-8149-4f06-a8c6-147bd62d8436-kube-api-access-mqq69\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886197    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-config\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886236    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-oauth-config\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886254    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-trusted-ca-bundle\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886275    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-service-ca\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886296 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886294    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-oauth-serving-cert\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.886674 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886536    8029 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 00:56:55.886674 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.886588    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c0051e9-8149-4f06-a8c6-147bd62d8436-tmp\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.887190 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.887112    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-config\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.887190 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.887155    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-service-ca\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.887190 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.887175    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-trusted-ca-bundle\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.887190 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.887194    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9c0051e9-8149-4f06-a8c6-147bd62d8436-oauth-serving-cert\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.888833 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.888805    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-oauth-config\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.889010 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.888991    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9c0051e9-8149-4f06-a8c6-147bd62d8436-console-serving-cert\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.895345 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.895325    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqq69\" (UniqueName: \"kubernetes.io/projected/9c0051e9-8149-4f06-a8c6-147bd62d8436-kube-api-access-mqq69\") pod \"console-6599b76c78-26thf\" (UID: \"9c0051e9-8149-4f06-a8c6-147bd62d8436\") " pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:55.989125 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:55.989105    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:56:56.288761 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:56.288697    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6599b76c78-26thf"]
Oct 31 00:56:56.292146 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:56:56.292116    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0051e9_8149_4f06_a8c6_147bd62d8436.slice/crio-ab38cc2043983402b6b7da007edd2e68279d2227d73720be5f78b157e7cea422 WatchSource:0}: Error finding container ab38cc2043983402b6b7da007edd2e68279d2227d73720be5f78b157e7cea422: Status 404 returned error can't find the container with id ab38cc2043983402b6b7da007edd2e68279d2227d73720be5f78b157e7cea422
Oct 31 00:56:56.303487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:56.303466    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6599b76c78-26thf" event={"ID":"9c0051e9-8149-4f06-a8c6-147bd62d8436","Type":"ContainerStarted","Data":"ab38cc2043983402b6b7da007edd2e68279d2227d73720be5f78b157e7cea422"}
Oct 31 00:56:57.311830 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:57.311795    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6599b76c78-26thf" event={"ID":"9c0051e9-8149-4f06-a8c6-147bd62d8436","Type":"ContainerStarted","Data":"fbdda9451a41e369fbdd7ddd3ef238699f3e195e5f021e10fe30f170741819f2"}
Oct 31 00:56:57.333128 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:57.333083    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6599b76c78-26thf" podStartSLOduration=2.3330703760000002 podStartE2EDuration="2.333070376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 00:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:56:57.331932266 +0000 UTC m=+14.319044242" watchObservedRunningTime="2026-10-31 00:56:57.333070376 +0000 UTC m=+14.320182349"
Oct 31 00:56:57.395085 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:57.395055    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:56:59.820682 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:59.820647    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:56:59.821037 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:56:59.820986    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"console-6db8b47f85-pkjfx\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") " pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:57:00.781759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:00.781724    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2z92c"
Oct 31 00:57:02.308259 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308216    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:57:02.308705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308504    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="prometheus" containerID="cri-o://d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce" gracePeriod=600
Oct 31 00:57:02.308705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308532    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy" containerID="cri-o://0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed" gracePeriod=600
Oct 31 00:57:02.308705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308579    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="thanos-sidecar" containerID="cri-o://e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256" gracePeriod=600
Oct 31 00:57:02.308705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308620    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-web" containerID="cri-o://2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad" gracePeriod=600
Oct 31 00:57:02.308705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308627    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d" gracePeriod=600
Oct 31 00:57:02.308705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.308676    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="config-reloader" containerID="cri-o://ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c" gracePeriod=600
Oct 31 00:57:02.572895 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.572865    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:02.743249 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743220    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743449 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743261    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743449 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743301    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743449 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743327    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743449 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743371    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743449 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743398    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743449 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743431    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743480    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743516    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743545    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743586    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743674    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743714    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743746    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743784    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.743809 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743810    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.744226 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743840    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.744226 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743885    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:57:02.744226 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743898    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.744226 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743949    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.744226 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.743977    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.744226 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.744029    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2db63efb-9319-4488-8589-2081d603b75e\" (UID: \"2db63efb-9319-4488-8589-2081d603b75e\") "
Oct 31 00:57:02.744517 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.744293    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.744517 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.744467    8029 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.744517 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.744488    8029 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-metrics-client-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.744517 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.744500    8029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-db\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.744875 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.744855    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.745382 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.745356    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.745757 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.745722    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.746713 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.746595    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.746811 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.746684    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.746873 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.746853    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "kube-api-access-vcmvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:02.747965 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.746827    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.747965 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.747259    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.748346 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.748313    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:02.748415 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.748333    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.748649 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.748624    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.748763 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.748653    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:57:02.748985 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.748955    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:02.750900 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.750643    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.750900 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.750783    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.752645 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.751173    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.762107 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.762087    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config" pod "2db63efb-9319-4488-8589-2081d603b75e" (UID: "2db63efb-9319-4488-8589-2081d603b75e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:02.845561 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845502    8029 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-tls-assets\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845561 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845527    8029 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-metrics-client-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845561 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845541    8029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845561 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845556    8029 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845594    8029 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-web-config\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845634    8029 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-kube-rbac-proxy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845648    8029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845663    8029 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845679    8029 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2db63efb-9319-4488-8589-2081d603b75e-config-out\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845692    8029 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845705    8029 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-grpc-tls\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845719    8029 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-config\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845733    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcmvc\" (UniqueName: \"kubernetes.io/projected/2db63efb-9319-4488-8589-2081d603b75e-kube-api-access-vcmvc\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845747    8029 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2db63efb-9319-4488-8589-2081d603b75e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845763    8029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845778    8029 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:02.845835 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:02.845792    8029 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2db63efb-9319-4488-8589-2081d603b75e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:03.340157 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340132    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d" exitCode=0
Oct 31 00:57:03.340157 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340157    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed" exitCode=0
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340163    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad" exitCode=0
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340169    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256" exitCode=0
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340175    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c" exitCode=0
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340181    8029 generic.go:356] "Generic (PLEG): container finished" podID="2db63efb-9319-4488-8589-2081d603b75e" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce" exitCode=0
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340201    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"}
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340241    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340291    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"}
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340332    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"}
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340354    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340373    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"}
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340414    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"}
Oct 31 00:57:03.340481 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340455    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"}
Oct 31 00:57:03.341054 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.340494    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2db63efb-9319-4488-8589-2081d603b75e","Type":"ContainerDied","Data":"d117dd70dea2f9f3b53647747cf4d9418508fa36daee5a57cfbdea807d5f3ddc"}
Oct 31 00:57:03.348330 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.348302    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.356948 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.356928    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.361920 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.361895    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:57:03.365269 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.365252    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.371256 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.371233    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:57:03.373023 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.372999    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.379740 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.379724    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.388715 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.388699    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.396468 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.396450    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.396727 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.396707    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.396793 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.396735    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"} err="failed to get container status \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist"
Oct 31 00:57:03.396793 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.396764    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.397065 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.397039    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.397145 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.397074    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"} err="failed to get container status \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist"
Oct 31 00:57:03.397145 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.397099    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.397411 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.397387    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.397500 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.397421    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"} err="failed to get container status \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist"
Oct 31 00:57:03.397500 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.397441    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.397780 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.397755    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.397871 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.397787    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"} err="failed to get container status \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist"
Oct 31 00:57:03.397871 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.397806    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.398051 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.398034    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.398086 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398058    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"} err="failed to get container status \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist"
Oct 31 00:57:03.398086 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398073    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.398304 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.398287    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.398341 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398310    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"} err="failed to get container status \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist"
Oct 31 00:57:03.398341 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398323    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.398567 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:03.398550    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.398635 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398570    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"} err="failed to get container status \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist"
Oct 31 00:57:03.398635 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398583    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.398832 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398816    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"} err="failed to get container status \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist"
Oct 31 00:57:03.398832 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.398831    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.399038 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399019    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"} err="failed to get container status \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist"
Oct 31 00:57:03.399038 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399037    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.399284 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399269    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"} err="failed to get container status \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist"
Oct 31 00:57:03.399284 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399283    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.399483 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399468    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"} err="failed to get container status \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist"
Oct 31 00:57:03.399483 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399483    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.399705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399684    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"} err="failed to get container status \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist"
Oct 31 00:57:03.399705 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399701    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.399915 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399898    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"} err="failed to get container status \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist"
Oct 31 00:57:03.399964 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.399915    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.400132 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400116    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"} err="failed to get container status \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist"
Oct 31 00:57:03.400132 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400131    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.400343 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400325    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"} err="failed to get container status \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist"
Oct 31 00:57:03.400396 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400342    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.400534 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400519    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"} err="failed to get container status \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist"
Oct 31 00:57:03.400582 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400534    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.400720 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400705    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"} err="failed to get container status \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist"
Oct 31 00:57:03.400761 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400719    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.400912 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400897    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"} err="failed to get container status \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist"
Oct 31 00:57:03.400956 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.400912    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.401123 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.401107    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"} err="failed to get container status \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist"
Oct 31 00:57:03.401167 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.401123    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.401318 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.401302    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"} err="failed to get container status \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist"
Oct 31 00:57:03.401362 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.401318    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.401492 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.401477    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"} err="failed to get container status \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist"
Oct 31 00:57:03.401531 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.401492    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.402047 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402031    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"} err="failed to get container status \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist"
Oct 31 00:57:03.402099 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402047    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.402236 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402219    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"} err="failed to get container status \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist"
Oct 31 00:57:03.402273 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402236    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.402413 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402398    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"} err="failed to get container status \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist"
Oct 31 00:57:03.402413 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402413    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.402584 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402569    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"} err="failed to get container status \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist"
Oct 31 00:57:03.402648 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402584    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.402774 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402759    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"} err="failed to get container status \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist"
Oct 31 00:57:03.402814 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402774    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.402940 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402925    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"} err="failed to get container status \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist"
Oct 31 00:57:03.402979 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.402940    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.403108 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403094    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"} err="failed to get container status \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist"
Oct 31 00:57:03.403143 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403107    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.403261 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403246    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"} err="failed to get container status \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist"
Oct 31 00:57:03.403300 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403261    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.403422 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403408    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"} err="failed to get container status \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist"
Oct 31 00:57:03.403460 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403421    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.403580 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403566    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"} err="failed to get container status \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist"
Oct 31 00:57:03.403642 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403580    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.403763 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403747    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"} err="failed to get container status \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist"
Oct 31 00:57:03.403805 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403763    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.403979 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403956    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"} err="failed to get container status \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist"
Oct 31 00:57:03.404031 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.403980    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.404188 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404173    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"} err="failed to get container status \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist"
Oct 31 00:57:03.404229 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404187    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.404382 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404366    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"} err="failed to get container status \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist"
Oct 31 00:57:03.404419 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404382    8029 scope.go:122] "RemoveContainer" containerID="b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"
Oct 31 00:57:03.404552 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404538    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d"} err="failed to get container status \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": rpc error: code = NotFound desc = could not find container \"b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d\": container with ID starting with b67f727e25d7d2b4c8f3179b46e004028bc3eb3607391408aa46796c02885c0d not found: ID does not exist"
Oct 31 00:57:03.404594 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404552    8029 scope.go:122] "RemoveContainer" containerID="0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"
Oct 31 00:57:03.404734 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404720    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed"} err="failed to get container status \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": rpc error: code = NotFound desc = could not find container \"0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed\": container with ID starting with 0ce828b5989d03eaadb0da1e378999cd26cea33f651331f688290dbfa724eeed not found: ID does not exist"
Oct 31 00:57:03.404734 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404733    8029 scope.go:122] "RemoveContainer" containerID="2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"
Oct 31 00:57:03.404890 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404874    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad"} err="failed to get container status \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": rpc error: code = NotFound desc = could not find container \"2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad\": container with ID starting with 2552d0e08147370f8c82ad16bb064d8d9dff8bf02849003d8322e2230ab770ad not found: ID does not exist"
Oct 31 00:57:03.404928 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.404889    8029 scope.go:122] "RemoveContainer" containerID="e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"
Oct 31 00:57:03.405048 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405032    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256"} err="failed to get container status \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": rpc error: code = NotFound desc = could not find container \"e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256\": container with ID starting with e32e4a905d0c2d7d4de822f8703b67e6fa4abc6ac1e81b5b30272a0b66429256 not found: ID does not exist"
Oct 31 00:57:03.405048 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405046    8029 scope.go:122] "RemoveContainer" containerID="ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"
Oct 31 00:57:03.405233 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405214    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c"} err="failed to get container status \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": rpc error: code = NotFound desc = could not find container \"ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c\": container with ID starting with ad443cd2860e683e2037b17780587b5f5dc2cc088ef2aa88a2e4afff2d937d8c not found: ID does not exist"
Oct 31 00:57:03.405303 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405235    8029 scope.go:122] "RemoveContainer" containerID="d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"
Oct 31 00:57:03.405423 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405403    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce"} err="failed to get container status \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": rpc error: code = NotFound desc = could not find container \"d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce\": container with ID starting with d77078558dacd19209fd356d7c13f277507c3251a899c184c7154927fe78a1ce not found: ID does not exist"
Oct 31 00:57:03.405423 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405421    8029 scope.go:122] "RemoveContainer" containerID="7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"
Oct 31 00:57:03.405621 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.405585    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331"} err="failed to get container status \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": rpc error: code = NotFound desc = could not find container \"7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331\": container with ID starting with 7366ddd9e608d3c7d18a945aced1153087e1d6e58c3fc63bada72c74edcbe331 not found: ID does not exist"
Oct 31 00:57:03.411269 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411246    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:57:03.411551 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411539    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-web"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411552    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-web"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411561    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-thanos"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411566    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-thanos"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411576    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="init-config-reloader"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411580    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="init-config-reloader"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411587    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="prometheus"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411592    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="prometheus"
Oct 31 00:57:03.411615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411614    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="config-reloader"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411621    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="config-reloader"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411634    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411639    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411649    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="thanos-sidecar"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411653    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="thanos-sidecar"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411712    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411720    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="thanos-sidecar"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411729    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-web"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411734    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="kube-rbac-proxy-thanos"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411740    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="config-reloader"
Oct 31 00:57:03.411879 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.411745    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2db63efb-9319-4488-8589-2081d603b75e" containerName="prometheus"
Oct 31 00:57:03.413989 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.413975    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.420314 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.420296    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 00:57:03.420409 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.420396    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-an58pn2d2p7d5\""
Oct 31 00:57:03.421065 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421045    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 00:57:03.421147 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421076    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 00:57:03.421292 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421278    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 00:57:03.421683 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421664    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 00:57:03.421683 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421672    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 00:57:03.421843 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421702    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 00:57:03.421843 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421707    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 00:57:03.421943 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421914    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 00:57:03.421943 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421922    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vtd8r\""
Oct 31 00:57:03.422015 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.421941    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 00:57:03.422970 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.422953    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 00:57:03.423045 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.422961    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 00:57:03.429167 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.429150    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 00:57:03.433290 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.429962    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 00:57:03.441421 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.441379    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:57:03.551027 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551002    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551027 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551037    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551208 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551055    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551208 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551076    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nwrw\" (UniqueName: \"kubernetes.io/projected/04279f09-aeff-4760-9418-c5f878b36e46-kube-api-access-9nwrw\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551270 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551218    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551270 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551247    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551270 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551264    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551377 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551280    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551377 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551333    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551456 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551429    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551496 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551455    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551496 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551476    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551496 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551492    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551665 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551589    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-web-config\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551665 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551638    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551723 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551671    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-config\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551723 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551694    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04279f09-aeff-4760-9418-c5f878b36e46-config-out\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551723 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551712    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551824 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551730    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.551824 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.551745    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04279f09-aeff-4760-9418-c5f878b36e46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652539 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652514    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652665 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652544    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-web-config\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652665 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652561    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652665 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652579    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-config\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652793 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652740    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04279f09-aeff-4760-9418-c5f878b36e46-config-out\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652793 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652779    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652884 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652813    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652884 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652840    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04279f09-aeff-4760-9418-c5f878b36e46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652965 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652884    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652965 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652918    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.652965 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652948    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653116 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.652979    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nwrw\" (UniqueName: \"kubernetes.io/projected/04279f09-aeff-4760-9418-c5f878b36e46-kube-api-access-9nwrw\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653116 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653033    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653116 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653063    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653116 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653092    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653116 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653114    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653371 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653141    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653371 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653181    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653371 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653234    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653371 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653269    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653760 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653402    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.653760 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653719    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.654229 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.653950    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.656301 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.656254    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-config\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.657007 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.656986    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04279f09-aeff-4760-9418-c5f878b36e46-config-out\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.657133 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.657077    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.657295 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.657262    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.657654 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.657624    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.658049 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.658019    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.658175 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.658075    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.658305 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.658281    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.658485 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.658457    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.658701 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.658679    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-web-config\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.658793 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.658682    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.659159 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.659137    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04279f09-aeff-4760-9418-c5f878b36e46-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.659239 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.659186    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.659276 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.659259    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.659445 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.659406    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04279f09-aeff-4760-9418-c5f878b36e46-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.659692 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.659674    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04279f09-aeff-4760-9418-c5f878b36e46-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.665615 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.665586    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nwrw\" (UniqueName: \"kubernetes.io/projected/04279f09-aeff-4760-9418-c5f878b36e46-kube-api-access-9nwrw\") pod \"prometheus-k8s-0\" (UID: \"04279f09-aeff-4760-9418-c5f878b36e46\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:03.732321 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:03.732276    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:04.021388 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:04.021360    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 00:57:04.024146 ip-10-0-140-244 kubenswrapper[8029]: W1031 00:57:04.024118    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04279f09_aeff_4760_9418_c5f878b36e46.slice/crio-59ab3d6ec0121eb72f37082993a33a10dab397122f3b532f912747cfbb86e166 WatchSource:0}: Error finding container 59ab3d6ec0121eb72f37082993a33a10dab397122f3b532f912747cfbb86e166: Status 404 returned error can't find the container with id 59ab3d6ec0121eb72f37082993a33a10dab397122f3b532f912747cfbb86e166
Oct 31 00:57:04.349060 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:04.348985    8029 generic.go:356] "Generic (PLEG): container finished" podID="04279f09-aeff-4760-9418-c5f878b36e46" containerID="63c35295f3dd31b8c8f8fc1483b62b0dea509f5fa1b73678637267e27f930088" exitCode=0
Oct 31 00:57:04.349060 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:04.349030    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerDied","Data":"63c35295f3dd31b8c8f8fc1483b62b0dea509f5fa1b73678637267e27f930088"}
Oct 31 00:57:04.349531 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:04.349093    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"59ab3d6ec0121eb72f37082993a33a10dab397122f3b532f912747cfbb86e166"}
Oct 31 00:57:05.180886 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.180858    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2db63efb-9319-4488-8589-2081d603b75e" path="/var/lib/kubelet/pods/2db63efb-9319-4488-8589-2081d603b75e/volumes"
Oct 31 00:57:05.355883 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.355851    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"c43879a58e64b20ee0fb88c158d7dc2be0bb508211c029d98f4fd2a1b559b28f"}
Oct 31 00:57:05.356228 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.355931    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"041513e9aa8abfaa80364bc01797333e6a70bc5ccdf18c562e76f221add0db3b"}
Oct 31 00:57:05.356228 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.355971    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"4b2a371822eda706fa54757d79fb2855e6e470a4aec3b07696683a2abeb9de3f"}
Oct 31 00:57:05.356228 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.356020    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"7c4469e6f083cb5bef43d2507bfd3ca5625c21ea6d952e97dd63ce13ac5d98ec"}
Oct 31 00:57:05.356228 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.356057    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"d000e5a925a144a2dccc82bfd48e4124b0d4c00fd70719eeb2fcbe76bc5bab02"}
Oct 31 00:57:05.356228 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.356089    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04279f09-aeff-4760-9418-c5f878b36e46","Type":"ContainerStarted","Data":"39abb4859dedd5c963c64a4f04c3b2825fa01166290bdda3836c0e3420839ec3"}
Oct 31 00:57:05.390240 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.390189    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.371597424 podStartE2EDuration="2.390175936s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.578512ms" isStatefulPod=false podCreationTimestamp="2026-10-31 00:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 00:57:05.38855134 +0000 UTC m=+22.375663330" watchObservedRunningTime="2026-10-31 00:57:05.390175936 +0000 UTC m=+22.377287912"
Oct 31 00:57:05.989616 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.989556    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:57:05.989775 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.989658    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:57:05.994289 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:05.994268    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:57:06.363722 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:06.363695    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6599b76c78-26thf"
Oct 31 00:57:06.445994 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:06.445967    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m9m29"
Oct 31 00:57:08.732839 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:08.732811    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:09.150825 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.150790    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6db8b47f85-pkjfx" podUID="fe4641e5-a8d7-4cad-b10b-0989e80736ae" containerName="console" containerID="cri-o://d20053c7b7f67ab34ecfd439975cfce7448853c7bafaaf15ad5e553a79719d7d" gracePeriod=15
Oct 31 00:57:09.371438 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.371403    8029 generic.go:356] "Generic (PLEG): container finished" podID="fe4641e5-a8d7-4cad-b10b-0989e80736ae" containerID="d20053c7b7f67ab34ecfd439975cfce7448853c7bafaaf15ad5e553a79719d7d" exitCode=0
Oct 31 00:57:09.371594 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.371479    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db8b47f85-pkjfx" event={"ID":"fe4641e5-a8d7-4cad-b10b-0989e80736ae","Type":"ContainerDied","Data":"d20053c7b7f67ab34ecfd439975cfce7448853c7bafaaf15ad5e553a79719d7d"}
Oct 31 00:57:09.390544 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.390524    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:57:09.405516 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405457    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405516 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405512    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405662 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405546    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405662 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405572    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405662 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405643    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405801 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405690    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405801 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405759    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.405801 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405765    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:09.405801 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405790    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") pod \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\" (UID: \"fe4641e5-a8d7-4cad-b10b-0989e80736ae\") "
Oct 31 00:57:09.406020 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.405993    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:09.406093 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.406068    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:09.406146 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.406085    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:57:09.406146 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.406081    8029 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-oauth-serving-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.406146 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.406123    8029 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-trusted-ca-bundle\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.406561 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.406539    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:09.407926 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.407904    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "kube-api-access-xllqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:09.408021 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.407964    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:09.408083 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.408027    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config" pod "fe4641e5-a8d7-4cad-b10b-0989e80736ae" (UID: "fe4641e5-a8d7-4cad-b10b-0989e80736ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:09.507334 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.507297    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xllqx\" (UniqueName: \"kubernetes.io/projected/fe4641e5-a8d7-4cad-b10b-0989e80736ae-kube-api-access-xllqx\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.507487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.507339    8029 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-serving-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.507487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.507358    8029 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-service-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.507487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.507373    8029 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4641e5-a8d7-4cad-b10b-0989e80736ae-tmp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.507487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.507389    8029 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-oauth-config\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:09.507487 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:09.507404    8029 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe4641e5-a8d7-4cad-b10b-0989e80736ae-console-config\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:10.376841 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:10.376812    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db8b47f85-pkjfx"
Oct 31 00:57:10.377279 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:10.376809    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db8b47f85-pkjfx" event={"ID":"fe4641e5-a8d7-4cad-b10b-0989e80736ae","Type":"ContainerDied","Data":"2941b4baa1289c44bd128dfa9314239c9935e1a690631b56c3caf51a3c92259d"}
Oct 31 00:57:10.377279 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:10.376941    8029 scope.go:122] "RemoveContainer" containerID="d20053c7b7f67ab34ecfd439975cfce7448853c7bafaaf15ad5e553a79719d7d"
Oct 31 00:57:10.397275 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:10.397247    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:57:10.403092 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:10.403072    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db8b47f85-pkjfx"]
Oct 31 00:57:11.181245 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:11.181207    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe4641e5-a8d7-4cad-b10b-0989e80736ae" path="/var/lib/kubelet/pods/fe4641e5-a8d7-4cad-b10b-0989e80736ae/volumes"
Oct 31 00:57:16.261759 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:16.261729    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rs9gd"
Oct 31 00:57:17.400174 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:17.400143    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6db4c97dcc-87jqk"
Oct 31 00:57:18.129148 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.129079    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5f44b4d54f-scl9x" podUID="6af5ed3a-6673-411b-87de-9ea59705bec6" containerName="console" containerID="cri-o://134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c" gracePeriod=15
Oct 31 00:57:18.401936 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.401884    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:57:18.410822 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.410799    8029 generic.go:356] "Generic (PLEG): container finished" podID="6af5ed3a-6673-411b-87de-9ea59705bec6" containerID="134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c" exitCode=0
Oct 31 00:57:18.410936 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.410869    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f44b4d54f-scl9x" event={"ID":"6af5ed3a-6673-411b-87de-9ea59705bec6","Type":"ContainerDied","Data":"134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c"}
Oct 31 00:57:18.410936 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.410911    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f44b4d54f-scl9x" event={"ID":"6af5ed3a-6673-411b-87de-9ea59705bec6","Type":"ContainerDied","Data":"1ab0b0b0c6609ffe6d99fa2e361655ec9ae60cf4cc4d324832edb3726ae12e95"}
Oct 31 00:57:18.410936 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.410925    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f44b4d54f-scl9x"
Oct 31 00:57:18.411072 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.410930    8029 scope.go:122] "RemoveContainer" containerID="134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c"
Oct 31 00:57:18.418779 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.418763    8029 scope.go:122] "RemoveContainer" containerID="134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c"
Oct 31 00:57:18.419088 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:18.419065    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c\": container with ID starting with 134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c not found: ID does not exist" containerID="134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c"
Oct 31 00:57:18.419162 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.419099    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c"} err="failed to get container status \"134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c\": rpc error: code = NotFound desc = could not find container \"134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c\": container with ID starting with 134e43c09157eaeca139bc633bdf8e9dcd2bcff3aa1046d34fb0de99fc36bc1c not found: ID does not exist"
Oct 31 00:57:18.477388 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477358    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477569 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477398    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477569 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477462    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477569 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477505    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477762 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477579    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477762 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477656    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477762 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477741    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") pod \"6af5ed3a-6673-411b-87de-9ea59705bec6\" (UID: \"6af5ed3a-6673-411b-87de-9ea59705bec6\") "
Oct 31 00:57:18.477936 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477826    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 00:57:18.477987 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.477916    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:18.478134 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.478085    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:18.478241 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.478195    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 00:57:18.478367 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.478316    8029 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-oauth-serving-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.478410 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.478376    8029 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af5ed3a-6673-411b-87de-9ea59705bec6-tmp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.478410 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.478391    8029 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-console-config\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.479794 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.479770    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "kube-api-access-tlc42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 00:57:18.480282 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.480259    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:18.480340 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.480303    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert" pod "6af5ed3a-6673-411b-87de-9ea59705bec6" (UID: "6af5ed3a-6673-411b-87de-9ea59705bec6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 00:57:18.578921 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.578888    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlc42\" (UniqueName: \"kubernetes.io/projected/6af5ed3a-6673-411b-87de-9ea59705bec6-kube-api-access-tlc42\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.578921 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.578913    8029 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-oauth-config\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.578921 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.578924    8029 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6af5ed3a-6673-411b-87de-9ea59705bec6-service-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.579147 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.578932    8029 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6af5ed3a-6673-411b-87de-9ea59705bec6-console-serving-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 00:57:18.732799 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.732767    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f44b4d54f-scl9x"]
Oct 31 00:57:18.737279 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:18.737256    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f44b4d54f-scl9x"]
Oct 31 00:57:19.180346 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:19.180313    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6af5ed3a-6673-411b-87de-9ea59705bec6" path="/var/lib/kubelet/pods/6af5ed3a-6673-411b-87de-9ea59705bec6/volumes"
Oct 31 00:57:19.440974 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:19.440894    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 00:57:45.453422 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:45.453387    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:45.469729 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:45.469707    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:45.523243 ip-10-0-140-244 kubenswrapper[8029]: I1031 00:57:45.523214    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 00:57:49.441696 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:57:49.441657    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 00:58:19.441416 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:58:19.441334    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 00:58:49.442126 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:58:49.442082    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 00:59:19.442083 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:59:19.442039    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 00:59:49.441678 ip-10-0-140-244 kubenswrapper[8029]: E1031 00:59:49.441582    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:00:19.440881 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:00:19.440846    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:00:49.441468 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:00:49.441429    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:01:19.441214 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:01:19.441131    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:01:43.193393 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:01:43.193361    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:01:43.203164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:01:43.203144    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:01:49.440760 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:01:49.440721    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:02:19.440902 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:02:19.440868    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:02:49.441199 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:02:49.441120    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:03:19.441788 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:03:19.441747    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:03:43.309733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.309700    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"]
Oct 31 01:03:43.310202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.310042    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fe4641e5-a8d7-4cad-b10b-0989e80736ae" containerName="console"
Oct 31 01:03:43.310202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.310055    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe4641e5-a8d7-4cad-b10b-0989e80736ae" containerName="console"
Oct 31 01:03:43.310202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.310069    8029 cpu_manager.go:414] "containerMap: removing container" podUID="6af5ed3a-6673-411b-87de-9ea59705bec6" containerName="console"
Oct 31 01:03:43.310202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.310075    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6af5ed3a-6673-411b-87de-9ea59705bec6" containerName="console"
Oct 31 01:03:43.310202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.310129    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe4641e5-a8d7-4cad-b10b-0989e80736ae" containerName="console"
Oct 31 01:03:43.310202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.310138    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6af5ed3a-6673-411b-87de-9ea59705bec6" containerName="console"
Oct 31 01:03:43.313263 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.313241    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.350792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.350755    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"]
Oct 31 01:03:43.378327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.378303    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 01:03:43.378460 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.378329    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 01:03:43.378460 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.378303    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 01:03:43.378460 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.378441    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 01:03:43.379498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.379477    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-tml79\""
Oct 31 01:03:43.379635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.379588    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 01:03:43.403839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.403811    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a98def7-65a7-4798-b8d6-73111762fe09-manifests\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.403954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.403843    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.403954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.403859    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.404055 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.404012    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.404055 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.404041    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbh9l\" (UniqueName: \"kubernetes.io/projected/6a98def7-65a7-4798-b8d6-73111762fe09-kube-api-access-mbh9l\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.404160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.404069    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a98def7-65a7-4798-b8d6-73111762fe09-tmp\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.504792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.504759    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.504952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.504798    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbh9l\" (UniqueName: \"kubernetes.io/projected/6a98def7-65a7-4798-b8d6-73111762fe09-kube-api-access-mbh9l\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.504952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.504926    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a98def7-65a7-4798-b8d6-73111762fe09-tmp\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.505070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.505042    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a98def7-65a7-4798-b8d6-73111762fe09-manifests\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.505133 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.505076    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.505133 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.505102    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.505411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.505392    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a98def7-65a7-4798-b8d6-73111762fe09-manifests\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.507618 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.507574    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.507719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.507620    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a98def7-65a7-4798-b8d6-73111762fe09-tmp\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.507765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.507718    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.507765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.507718    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a98def7-65a7-4798-b8d6-73111762fe09-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.513014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.512992    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbh9l\" (UniqueName: \"kubernetes.io/projected/6a98def7-65a7-4798-b8d6-73111762fe09-kube-api-access-mbh9l\") pod \"opendatahub-operator-controller-manager-7f58f8c74c-5b8p5\" (UID: \"6a98def7-65a7-4798-b8d6-73111762fe09\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.627432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.627411    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:43.933680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.933588    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"]
Oct 31 01:03:43.937099 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:03:43.937068    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a98def7_65a7_4798_b8d6_73111762fe09.slice/crio-0b7ce69b5aafa12e70cfb7d7dbd336325f213c6d6a4237a0d84a4c990f2f4772 WatchSource:0}: Error finding container 0b7ce69b5aafa12e70cfb7d7dbd336325f213c6d6a4237a0d84a4c990f2f4772: Status 404 returned error can't find the container with id 0b7ce69b5aafa12e70cfb7d7dbd336325f213c6d6a4237a0d84a4c990f2f4772
Oct 31 01:03:43.938726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:43.938708    8029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:03:44.784815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:44.784767    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5" event={"ID":"6a98def7-65a7-4798-b8d6-73111762fe09","Type":"ContainerStarted","Data":"0b7ce69b5aafa12e70cfb7d7dbd336325f213c6d6a4237a0d84a4c990f2f4772"}
Oct 31 01:03:48.853320 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:48.853287    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"]
Oct 31 01:03:48.856666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:48.856651    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:48.874280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:48.874257    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-8rdfd\""
Oct 31 01:03:48.874383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:48.874288    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"]
Oct 31 01:03:48.957566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:48.957536    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 31 01:03:48.957566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:48.957564    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 31 01:03:49.050856 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.050826    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqc27\" (UniqueName: \"kubernetes.io/projected/6be48609-5359-44ac-b7f5-d8872cbdec93-kube-api-access-zqc27\") pod \"jobset-operator-674cd59dc5-hdssj\" (UID: \"6be48609-5359-44ac-b7f5-d8872cbdec93\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.051035 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.051010    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6be48609-5359-44ac-b7f5-d8872cbdec93-tmp\") pod \"jobset-operator-674cd59dc5-hdssj\" (UID: \"6be48609-5359-44ac-b7f5-d8872cbdec93\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.152553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.152476    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqc27\" (UniqueName: \"kubernetes.io/projected/6be48609-5359-44ac-b7f5-d8872cbdec93-kube-api-access-zqc27\") pod \"jobset-operator-674cd59dc5-hdssj\" (UID: \"6be48609-5359-44ac-b7f5-d8872cbdec93\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.152729 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.152582    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6be48609-5359-44ac-b7f5-d8872cbdec93-tmp\") pod \"jobset-operator-674cd59dc5-hdssj\" (UID: \"6be48609-5359-44ac-b7f5-d8872cbdec93\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.152971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.152949    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6be48609-5359-44ac-b7f5-d8872cbdec93-tmp\") pod \"jobset-operator-674cd59dc5-hdssj\" (UID: \"6be48609-5359-44ac-b7f5-d8872cbdec93\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.160755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.160721    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqc27\" (UniqueName: \"kubernetes.io/projected/6be48609-5359-44ac-b7f5-d8872cbdec93-kube-api-access-zqc27\") pod \"jobset-operator-674cd59dc5-hdssj\" (UID: \"6be48609-5359-44ac-b7f5-d8872cbdec93\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.180105 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.180075    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"
Oct 31 01:03:49.444929 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:03:49.444855    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:03:49.467872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.467849    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj"]
Oct 31 01:03:49.470265 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:03:49.470236    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6be48609_5359_44ac_b7f5_d8872cbdec93.slice/crio-1e095bf1af877f575253d034eda2a709d7bbc1af1772da169dbf281bff2c135a WatchSource:0}: Error finding container 1e095bf1af877f575253d034eda2a709d7bbc1af1772da169dbf281bff2c135a: Status 404 returned error can't find the container with id 1e095bf1af877f575253d034eda2a709d7bbc1af1772da169dbf281bff2c135a
Oct 31 01:03:49.809223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:49.809134    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj" event={"ID":"6be48609-5359-44ac-b7f5-d8872cbdec93","Type":"ContainerStarted","Data":"1e095bf1af877f575253d034eda2a709d7bbc1af1772da169dbf281bff2c135a"}
Oct 31 01:03:50.817498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:50.817404    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5" event={"ID":"6a98def7-65a7-4798-b8d6-73111762fe09","Type":"ContainerStarted","Data":"df1d2f6e9efa244bcc172800406431c41bba5a312a414368a167c8b22c013349"}
Oct 31 01:03:51.824480 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:51.824428    8029 generic.go:356] "Generic (PLEG): container finished" podID="6a98def7-65a7-4798-b8d6-73111762fe09" containerID="df1d2f6e9efa244bcc172800406431c41bba5a312a414368a167c8b22c013349" exitCode=0
Oct 31 01:03:51.824919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:51.824498    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5" event={"ID":"6a98def7-65a7-4798-b8d6-73111762fe09","Type":"ContainerDied","Data":"df1d2f6e9efa244bcc172800406431c41bba5a312a414368a167c8b22c013349"}
Oct 31 01:03:52.831378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:52.831344    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj" event={"ID":"6be48609-5359-44ac-b7f5-d8872cbdec93","Type":"ContainerStarted","Data":"3dae301951374aed614f355a12e29135cadfb42918c0851ae457ffec73bc1f70"}
Oct 31 01:03:52.833526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:52.833499    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5" event={"ID":"6a98def7-65a7-4798-b8d6-73111762fe09","Type":"ContainerStarted","Data":"41f88e4ddeedcf82191d249fc375c32167dc6b1caebc062654d43379898c6a44"}
Oct 31 01:03:52.833677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:52.833658    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:03:52.851775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:52.851730    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-hdssj" podStartSLOduration=1.640019554 podStartE2EDuration="4.851696897s" totalImagesPullingTime="3.211677343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:03:52.851517494 +0000 UTC m=+429.838629469" watchObservedRunningTime="2026-10-31 01:03:52.851696897 +0000 UTC m=+429.838808870"
Oct 31 01:03:52.889951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:03:52.889904    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5" podStartSLOduration=2.218018858 podStartE2EDuration="9.889893206s" totalImagesPullingTime="7.051303519s" totalInitContainerRuntime="620.570829ms" isStatefulPod=false podCreationTimestamp="2026-10-31 01:03:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:03:52.888073344 +0000 UTC m=+429.875185322" watchObservedRunningTime="2026-10-31 01:03:52.889893206 +0000 UTC m=+429.877005173"
Oct 31 01:04:03.841087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:03.841057    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f58f8c74c-5b8p5"
Oct 31 01:04:06.965205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:06.965174    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"]
Oct 31 01:04:06.972066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:06.972043    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:06.988359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:06.988332    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 31 01:04:06.988359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:06.988348    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 31 01:04:06.988359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:06.988337    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ppbqq\""
Oct 31 01:04:06.989566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:06.989540    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"]
Oct 31 01:04:07.008253 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.008227    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw45h\" (UniqueName: \"kubernetes.io/projected/eeda0349-fac8-42fc-a73d-f81c06888419-kube-api-access-hw45h\") pod \"cert-manager-webhook-5dbbbf75cc-q2rrd\" (UID: \"eeda0349-fac8-42fc-a73d-f81c06888419\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.008374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.008357    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eeda0349-fac8-42fc-a73d-f81c06888419-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-q2rrd\" (UID: \"eeda0349-fac8-42fc-a73d-f81c06888419\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.109766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.109733    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eeda0349-fac8-42fc-a73d-f81c06888419-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-q2rrd\" (UID: \"eeda0349-fac8-42fc-a73d-f81c06888419\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.109879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.109778    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw45h\" (UniqueName: \"kubernetes.io/projected/eeda0349-fac8-42fc-a73d-f81c06888419-kube-api-access-hw45h\") pod \"cert-manager-webhook-5dbbbf75cc-q2rrd\" (UID: \"eeda0349-fac8-42fc-a73d-f81c06888419\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.140719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.140693    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eeda0349-fac8-42fc-a73d-f81c06888419-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-q2rrd\" (UID: \"eeda0349-fac8-42fc-a73d-f81c06888419\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.140922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.140901    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw45h\" (UniqueName: \"kubernetes.io/projected/eeda0349-fac8-42fc-a73d-f81c06888419-kube-api-access-hw45h\") pod \"cert-manager-webhook-5dbbbf75cc-q2rrd\" (UID: \"eeda0349-fac8-42fc-a73d-f81c06888419\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.283750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.283688    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:07.581536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.581475    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"]
Oct 31 01:04:07.585088 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:04:07.585058    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeeda0349_fac8_42fc_a73d_f81c06888419.slice/crio-f6c5ddbeeeba3a1d34d0a56eb44cb5afa6fc5e3f784673015db34c8c77581d03 WatchSource:0}: Error finding container f6c5ddbeeeba3a1d34d0a56eb44cb5afa6fc5e3f784673015db34c8c77581d03: Status 404 returned error can't find the container with id f6c5ddbeeeba3a1d34d0a56eb44cb5afa6fc5e3f784673015db34c8c77581d03
Oct 31 01:04:07.899203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:07.899171    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" event={"ID":"eeda0349-fac8-42fc-a73d-f81c06888419","Type":"ContainerStarted","Data":"f6c5ddbeeeba3a1d34d0a56eb44cb5afa6fc5e3f784673015db34c8c77581d03"}
Oct 31 01:04:11.930972 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:11.930932    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" event={"ID":"eeda0349-fac8-42fc-a73d-f81c06888419","Type":"ContainerStarted","Data":"786552b12d94d97497912b6958462b77a16621f6156c097ed88dc00bcecd6fb1"}
Oct 31 01:04:11.931420 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:11.931013    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:13.438031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.437979    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" podStartSLOduration=3.813142556 podStartE2EDuration="7.437964887s" totalImagesPullingTime="3.624822331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:11.952580359 +0000 UTC m=+448.939692336" watchObservedRunningTime="2026-10-31 01:04:13.437964887 +0000 UTC m=+450.425076862"
Oct 31 01:04:13.438783 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.438765    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"]
Oct 31 01:04:13.442110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.442095    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.445345 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.445328    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-spbm4\""
Oct 31 01:04:13.450111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.450088    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"]
Oct 31 01:04:13.566298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.566267    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c29f64eb-5fce-43fa-afb1-46be82ac101e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-g9kzq\" (UID: \"c29f64eb-5fce-43fa-afb1-46be82ac101e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.566417 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.566322    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcf2l\" (UniqueName: \"kubernetes.io/projected/c29f64eb-5fce-43fa-afb1-46be82ac101e-kube-api-access-zcf2l\") pod \"cert-manager-cainjector-5c57bbd4fc-g9kzq\" (UID: \"c29f64eb-5fce-43fa-afb1-46be82ac101e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.667373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.667341    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c29f64eb-5fce-43fa-afb1-46be82ac101e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-g9kzq\" (UID: \"c29f64eb-5fce-43fa-afb1-46be82ac101e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.667504 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.667414    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcf2l\" (UniqueName: \"kubernetes.io/projected/c29f64eb-5fce-43fa-afb1-46be82ac101e-kube-api-access-zcf2l\") pod \"cert-manager-cainjector-5c57bbd4fc-g9kzq\" (UID: \"c29f64eb-5fce-43fa-afb1-46be82ac101e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.679228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.679198    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c29f64eb-5fce-43fa-afb1-46be82ac101e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-g9kzq\" (UID: \"c29f64eb-5fce-43fa-afb1-46be82ac101e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.679433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.679411    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcf2l\" (UniqueName: \"kubernetes.io/projected/c29f64eb-5fce-43fa-afb1-46be82ac101e-kube-api-access-zcf2l\") pod \"cert-manager-cainjector-5c57bbd4fc-g9kzq\" (UID: \"c29f64eb-5fce-43fa-afb1-46be82ac101e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:13.754745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:13.754684    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"
Oct 31 01:04:14.031629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:14.031531    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq"]
Oct 31 01:04:14.034753 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:04:14.034727    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc29f64eb_5fce_43fa_afb1_46be82ac101e.slice/crio-117d170bbf896d1a2866d368424e732113a10b66dbc64cf68486cebf80d5dedf WatchSource:0}: Error finding container 117d170bbf896d1a2866d368424e732113a10b66dbc64cf68486cebf80d5dedf: Status 404 returned error can't find the container with id 117d170bbf896d1a2866d368424e732113a10b66dbc64cf68486cebf80d5dedf
Oct 31 01:04:14.952461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:14.952428    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq" event={"ID":"c29f64eb-5fce-43fa-afb1-46be82ac101e","Type":"ContainerStarted","Data":"b53bcd2b7344a7a96586a7d928c9e5d3090cb164d64711fa63953d05022311f2"}
Oct 31 01:04:14.952893 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:14.952471    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq" event={"ID":"c29f64eb-5fce-43fa-afb1-46be82ac101e","Type":"ContainerStarted","Data":"117d170bbf896d1a2866d368424e732113a10b66dbc64cf68486cebf80d5dedf"}
Oct 31 01:04:14.989475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:14.989419    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-g9kzq" podStartSLOduration=1.9894050970000001 podStartE2EDuration="1.989405097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:14.988862974 +0000 UTC m=+451.975974948" watchObservedRunningTime="2026-10-31 01:04:14.989405097 +0000 UTC m=+451.976517072"
Oct 31 01:04:17.466894 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.466858    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-qvl5p"]
Oct 31 01:04:17.470319 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.470302    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.479939 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.479919    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-h6tjp\""
Oct 31 01:04:17.488197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.488172    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-qvl5p"]
Oct 31 01:04:17.602299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.602271    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b76ce06-4160-47c2-b3c4-bbb9941bc02a-bound-sa-token\") pod \"cert-manager-597b95cb84-qvl5p\" (UID: \"5b76ce06-4160-47c2-b3c4-bbb9941bc02a\") " pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.602440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.602315    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jln6h\" (UniqueName: \"kubernetes.io/projected/5b76ce06-4160-47c2-b3c4-bbb9941bc02a-kube-api-access-jln6h\") pod \"cert-manager-597b95cb84-qvl5p\" (UID: \"5b76ce06-4160-47c2-b3c4-bbb9941bc02a\") " pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.703587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.703549    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b76ce06-4160-47c2-b3c4-bbb9941bc02a-bound-sa-token\") pod \"cert-manager-597b95cb84-qvl5p\" (UID: \"5b76ce06-4160-47c2-b3c4-bbb9941bc02a\") " pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.703734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.703625    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jln6h\" (UniqueName: \"kubernetes.io/projected/5b76ce06-4160-47c2-b3c4-bbb9941bc02a-kube-api-access-jln6h\") pod \"cert-manager-597b95cb84-qvl5p\" (UID: \"5b76ce06-4160-47c2-b3c4-bbb9941bc02a\") " pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.714790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.714767    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jln6h\" (UniqueName: \"kubernetes.io/projected/5b76ce06-4160-47c2-b3c4-bbb9941bc02a-kube-api-access-jln6h\") pod \"cert-manager-597b95cb84-qvl5p\" (UID: \"5b76ce06-4160-47c2-b3c4-bbb9941bc02a\") " pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.717638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.717586    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b76ce06-4160-47c2-b3c4-bbb9941bc02a-bound-sa-token\") pod \"cert-manager-597b95cb84-qvl5p\" (UID: \"5b76ce06-4160-47c2-b3c4-bbb9941bc02a\") " pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.781296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.781271    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-qvl5p"
Oct 31 01:04:17.937207 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:17.937178    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd"
Oct 31 01:04:18.059328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:18.059258    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-qvl5p"]
Oct 31 01:04:18.062786 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:04:18.062756    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b76ce06_4160_47c2_b3c4_bbb9941bc02a.slice/crio-1aa79fde8c1f1a7bfe900727351b5fc4666792956a6271fddd43186e465f864c WatchSource:0}: Error finding container 1aa79fde8c1f1a7bfe900727351b5fc4666792956a6271fddd43186e465f864c: Status 404 returned error can't find the container with id 1aa79fde8c1f1a7bfe900727351b5fc4666792956a6271fddd43186e465f864c
Oct 31 01:04:18.975651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:18.975556    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-qvl5p" event={"ID":"5b76ce06-4160-47c2-b3c4-bbb9941bc02a","Type":"ContainerStarted","Data":"0cdb0aa60d780cd7912468769975029ecd9c21f87d02dccbca25676e85ca6fd8"}
Oct 31 01:04:18.975651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:18.975616    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-qvl5p" event={"ID":"5b76ce06-4160-47c2-b3c4-bbb9941bc02a","Type":"ContainerStarted","Data":"1aa79fde8c1f1a7bfe900727351b5fc4666792956a6271fddd43186e465f864c"}
Oct 31 01:04:18.994388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:18.994342    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-qvl5p" podStartSLOduration=1.9943271820000001 podStartE2EDuration="1.994327182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:18.993570069 +0000 UTC m=+455.980682045" watchObservedRunningTime="2026-10-31 01:04:18.994327182 +0000 UTC m=+455.981439161"
Oct 31 01:04:19.441269 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:04:19.441233    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:04:20.872645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.872582    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-858778b8-88xjg"]
Oct 31 01:04:20.877325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.877304    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:20.889096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.886193    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 31 01:04:20.889096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.887054    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 31 01:04:20.889096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.887072    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tqk4d\""
Oct 31 01:04:20.889096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.888170    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 01:04:20.889096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.888251    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 01:04:20.890206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:20.890186    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-858778b8-88xjg"]
Oct 31 01:04:21.036348 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.036313    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h5k5\" (UniqueName: \"kubernetes.io/projected/1e413599-cc6e-4805-85f4-c27406052466-kube-api-access-4h5k5\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.036539 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.036368    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e413599-cc6e-4805-85f4-c27406052466-tls-certs\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.036539 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.036396    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e413599-cc6e-4805-85f4-c27406052466-tmp\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.137454 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.137383    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h5k5\" (UniqueName: \"kubernetes.io/projected/1e413599-cc6e-4805-85f4-c27406052466-kube-api-access-4h5k5\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.137454 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.137424    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e413599-cc6e-4805-85f4-c27406052466-tls-certs\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.137454 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.137448    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e413599-cc6e-4805-85f4-c27406052466-tmp\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.139821 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.139795    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e413599-cc6e-4805-85f4-c27406052466-tmp\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.140052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.140030    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e413599-cc6e-4805-85f4-c27406052466-tls-certs\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.147415 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.147382    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h5k5\" (UniqueName: \"kubernetes.io/projected/1e413599-cc6e-4805-85f4-c27406052466-kube-api-access-4h5k5\") pod \"kube-auth-proxy-858778b8-88xjg\" (UID: \"1e413599-cc6e-4805-85f4-c27406052466\") " pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.204496 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.204469    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg"
Oct 31 01:04:21.497633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.497537    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-858778b8-88xjg"]
Oct 31 01:04:21.501696 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:04:21.501668    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e413599_cc6e_4805_85f4_c27406052466.slice/crio-fc3d400e750ea5e937e6988e217947c290b752f6f2dddf1d2d9ce5c263851ff6 WatchSource:0}: Error finding container fc3d400e750ea5e937e6988e217947c290b752f6f2dddf1d2d9ce5c263851ff6: Status 404 returned error can't find the container with id fc3d400e750ea5e937e6988e217947c290b752f6f2dddf1d2d9ce5c263851ff6
Oct 31 01:04:21.991058 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:21.991027    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg" event={"ID":"1e413599-cc6e-4805-85f4-c27406052466","Type":"ContainerStarted","Data":"fc3d400e750ea5e937e6988e217947c290b752f6f2dddf1d2d9ce5c263851ff6"}
Oct 31 01:04:27.620129 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.620091    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:27.626323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.626302    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.631732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.631711    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-22f5d\""
Oct 31 01:04:27.635862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.635839    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 31 01:04:27.635862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.635867    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 01:04:27.636038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.635876    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 01:04:27.644640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.644594    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:27.728722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.728689    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:27.801515 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.801481    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk594\" (UniqueName: \"kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594\") pod \"limitador-limitador-74dfcfb894-tm4fg\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.801729 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.801637    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file\") pod \"limitador-limitador-74dfcfb894-tm4fg\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.903081 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.902993    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file\") pod \"limitador-limitador-74dfcfb894-tm4fg\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.903247 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.903110    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk594\" (UniqueName: \"kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594\") pod \"limitador-limitador-74dfcfb894-tm4fg\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.903799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.903772    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file\") pod \"limitador-limitador-74dfcfb894-tm4fg\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.923020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.922995    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk594\" (UniqueName: \"kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594\") pod \"limitador-limitador-74dfcfb894-tm4fg\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:27.941278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:27.941254    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:28.250621 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:28.250523    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:28.254441 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:04:28.254409    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5109bf1c_457e_4b46_9e77_7f1f5e3311d1.slice/crio-02bcfdbe424a3b046be0b64fe96401df12e3f32c662068ff4b43510652cfb582 WatchSource:0}: Error finding container 02bcfdbe424a3b046be0b64fe96401df12e3f32c662068ff4b43510652cfb582: Status 404 returned error can't find the container with id 02bcfdbe424a3b046be0b64fe96401df12e3f32c662068ff4b43510652cfb582
Oct 31 01:04:29.023614 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.023570    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg" event={"ID":"5109bf1c-457e-4b46-9e77-7f1f5e3311d1","Type":"ContainerStarted","Data":"02bcfdbe424a3b046be0b64fe96401df12e3f32c662068ff4b43510652cfb582"}
Oct 31 01:04:29.025104 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.025068    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg" event={"ID":"1e413599-cc6e-4805-85f4-c27406052466","Type":"ContainerStarted","Data":"0784448ccaa1c60d5df3e0b5877967183f29c3abd0fb0479b658925b9a5c0191"}
Oct 31 01:04:29.056595 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.056541    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-858778b8-88xjg" podStartSLOduration=2.154643053 podStartE2EDuration="9.056523801s" totalImagesPullingTime="6.901880748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:29.054906098 +0000 UTC m=+466.042018073" watchObservedRunningTime="2026-10-31 01:04:29.056523801 +0000 UTC m=+466.043635777"
Oct 31 01:04:29.295009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.294933    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hh27j"]
Oct 31 01:04:29.298280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.298263    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:04:29.303519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.303497    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9t9bz\""
Oct 31 01:04:29.317050 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.317024    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hh27j"]
Oct 31 01:04:29.416857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.416824    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kb9j\" (UniqueName: \"kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j\") pod \"authorino-56c6c9ddd5-hh27j\" (UID: \"d8f360ff-3c50-4dc5-af9e-19058b40b72b\") " pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:04:29.517533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.517490    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kb9j\" (UniqueName: \"kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j\") pod \"authorino-56c6c9ddd5-hh27j\" (UID: \"d8f360ff-3c50-4dc5-af9e-19058b40b72b\") " pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:04:29.537629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.532617    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kb9j\" (UniqueName: \"kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j\") pod \"authorino-56c6c9ddd5-hh27j\" (UID: \"d8f360ff-3c50-4dc5-af9e-19058b40b72b\") " pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:04:29.609537 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.609500    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:04:29.920841 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:29.920762    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hh27j"]
Oct 31 01:04:30.031107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:30.031061    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j" event={"ID":"d8f360ff-3c50-4dc5-af9e-19058b40b72b","Type":"ContainerStarted","Data":"d57cb8358bc6e8c2d68d516d9c04a77647cfe04f105f7133ef3c28f6ddea2381"}
Oct 31 01:04:32.043644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:32.043614    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg" event={"ID":"5109bf1c-457e-4b46-9e77-7f1f5e3311d1","Type":"ContainerStarted","Data":"680d6af8093602d0e2e55fef169ed4b2d65b896835c14c430c5447441b1d559e"}
Oct 31 01:04:32.044042 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:32.043861    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:32.068141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:32.068096    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg" podStartSLOduration=1.7643910329999999 podStartE2EDuration="5.068082823s" totalImagesPullingTime="3.30369179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:32.066384008 +0000 UTC m=+469.053495983" watchObservedRunningTime="2026-10-31 01:04:32.068082823 +0000 UTC m=+469.055194811"
Oct 31 01:04:33.049414 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:33.049374    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j" event={"ID":"d8f360ff-3c50-4dc5-af9e-19058b40b72b","Type":"ContainerStarted","Data":"6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f"}
Oct 31 01:04:33.068505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:33.067817    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j" podStartSLOduration=1.06467479 podStartE2EDuration="4.067798841s" totalImagesPullingTime="3.003124051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:33.067198234 +0000 UTC m=+470.054310211" watchObservedRunningTime="2026-10-31 01:04:33.067798841 +0000 UTC m=+470.054910818"
Oct 31 01:04:42.552172 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:42.552139    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:42.552583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:42.552319    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg" podUID="5109bf1c-457e-4b46-9e77-7f1f5e3311d1" containerName="limitador" containerID="cri-o://680d6af8093602d0e2e55fef169ed4b2d65b896835c14c430c5447441b1d559e" gracePeriod=30
Oct 31 01:04:42.553027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:42.553006    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:43.093238 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.093207    8029 generic.go:356] "Generic (PLEG): container finished" podID="5109bf1c-457e-4b46-9e77-7f1f5e3311d1" containerID="680d6af8093602d0e2e55fef169ed4b2d65b896835c14c430c5447441b1d559e" exitCode=0
Oct 31 01:04:43.093381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.093284    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg" event={"ID":"5109bf1c-457e-4b46-9e77-7f1f5e3311d1","Type":"ContainerDied","Data":"680d6af8093602d0e2e55fef169ed4b2d65b896835c14c430c5447441b1d559e"}
Oct 31 01:04:43.542990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.542967    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:43.548077 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.548056    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file\" (UniqueName: \"kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file\") pod \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") "
Oct 31 01:04:43.548188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.548133    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594\" (UniqueName: \"kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594\") pod \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\" (UID: \"5109bf1c-457e-4b46-9e77-7f1f5e3311d1\") "
Oct 31 01:04:43.548367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.548344    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file" pod "5109bf1c-457e-4b46-9e77-7f1f5e3311d1" (UID: "5109bf1c-457e-4b46-9e77-7f1f5e3311d1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:04:43.550253 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.550234    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594" pod "5109bf1c-457e-4b46-9e77-7f1f5e3311d1" (UID: "5109bf1c-457e-4b46-9e77-7f1f5e3311d1"). InnerVolumeSpecName "kube-api-access-rk594". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:04:43.648905 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.648835    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk594\" (UniqueName: \"kubernetes.io/projected/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-kube-api-access-rk594\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:43.648905 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:43.648863    8029 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5109bf1c-457e-4b46-9e77-7f1f5e3311d1-config-file\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:04:44.099098 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:44.099004    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg" event={"ID":"5109bf1c-457e-4b46-9e77-7f1f5e3311d1","Type":"ContainerDied","Data":"02bcfdbe424a3b046be0b64fe96401df12e3f32c662068ff4b43510652cfb582"}
Oct 31 01:04:44.099098 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:44.099032    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"
Oct 31 01:04:44.099098 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:44.099094    8029 scope.go:122] "RemoveContainer" containerID="680d6af8093602d0e2e55fef169ed4b2d65b896835c14c430c5447441b1d559e"
Oct 31 01:04:44.119744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:44.119719    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:44.124150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:44.124127    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-tm4fg"]
Oct 31 01:04:45.181474 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:45.181437    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5109bf1c-457e-4b46-9e77-7f1f5e3311d1" path="/var/lib/kubelet/pods/5109bf1c-457e-4b46-9e77-7f1f5e3311d1/volumes"
Oct 31 01:04:49.441767 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:04:49.441732    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:04:50.069041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.069010    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"]
Oct 31 01:04:50.069386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.069373    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5109bf1c-457e-4b46-9e77-7f1f5e3311d1" containerName="limitador"
Oct 31 01:04:50.069386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.069385    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5109bf1c-457e-4b46-9e77-7f1f5e3311d1" containerName="limitador"
Oct 31 01:04:50.069515 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.069451    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5109bf1c-457e-4b46-9e77-7f1f5e3311d1" containerName="limitador"
Oct 31 01:04:50.073535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.073515    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.090099 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.090076    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 31 01:04:50.093487 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.093464    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 31 01:04:50.093487 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.093473    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-5zrcf\""
Oct 31 01:04:50.096049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.096026    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"]
Oct 31 01:04:50.101008 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.100991    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 31 01:04:50.106503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.106481    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5b5k\" (UniqueName: \"kubernetes.io/projected/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-kube-api-access-p5b5k\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.106613 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.106511    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-cert\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.106721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.106702    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-metrics-certs\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.106806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.106769    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-manager-config\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.208069 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.208018    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-manager-config\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.208289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.208136    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5b5k\" (UniqueName: \"kubernetes.io/projected/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-kube-api-access-p5b5k\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.208289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.208172    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-cert\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.208289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.208260    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-metrics-certs\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.208874 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.208847    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-manager-config\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.211590 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.211563    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-metrics-certs\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.211590 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.211578    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-cert\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.224969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.224943    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5b5k\" (UniqueName: \"kubernetes.io/projected/29aaf902-59b6-41f3-9f3e-2af1e9e9fec6-kube-api-access-p5b5k\") pod \"jobset-controller-manager-5d466cf57f-6xz5v\" (UID: \"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6\") " pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.386012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.385976    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:50.663312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:50.663081    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"]
Oct 31 01:04:50.666056 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:04:50.666024    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29aaf902_59b6_41f3_9f3e_2af1e9e9fec6.slice/crio-e1051e2f850d9e526ea9f0d3ec1bff3eb2d0c809316e051b6c4d650854447fdd WatchSource:0}: Error finding container e1051e2f850d9e526ea9f0d3ec1bff3eb2d0c809316e051b6c4d650854447fdd: Status 404 returned error can't find the container with id e1051e2f850d9e526ea9f0d3ec1bff3eb2d0c809316e051b6c4d650854447fdd
Oct 31 01:04:51.129635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:51.129586    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v" event={"ID":"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6","Type":"ContainerStarted","Data":"e1051e2f850d9e526ea9f0d3ec1bff3eb2d0c809316e051b6c4d650854447fdd"}
Oct 31 01:04:55.149398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:55.149365    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v" event={"ID":"29aaf902-59b6-41f3-9f3e-2af1e9e9fec6","Type":"ContainerStarted","Data":"9fc74c07ec8209ed6b885cd4ecf726c0bf57011b728f52d2c6e9061f0f29b754"}
Oct 31 01:04:55.149873 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:55.149494    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:04:55.166369 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:04:55.166326    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v" podStartSLOduration=1.46674511 podStartE2EDuration="5.166312187s" totalImagesPullingTime="3.699567077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:04:55.165094796 +0000 UTC m=+492.152206772" watchObservedRunningTime="2026-10-31 01:04:55.166312187 +0000 UTC m=+492.153424227"
Oct 31 01:05:06.158687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:06.158659    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-5d466cf57f-6xz5v"
Oct 31 01:05:19.440670 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:05:19.440634    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:05:20.343378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.343346    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gmb5t"]
Oct 31 01:05:20.347458 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.347437    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.370377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.370346    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 01:05:20.370513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.370420    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 01:05:20.371536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.371518    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 31 01:05:20.371536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.371523    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-5p52d\""
Oct 31 01:05:20.385748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.385724    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gmb5t"]
Oct 31 01:05:20.431456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.431428    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-nmdfz"]
Oct 31 01:05:20.434842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.434827    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.468512 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.468488    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 31 01:05:20.469135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.468488    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 31 01:05:20.469135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.468707    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6nkdn\""
Oct 31 01:05:20.469595 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.469578    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-nmdfz"]
Oct 31 01:05:20.483823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.483802    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5l6c\" (UniqueName: \"kubernetes.io/projected/65e086a0-1eb5-46cf-895e-8a9cd892ce6b-kube-api-access-w5l6c\") pod \"model-serving-api-75bd89fdfc-gmb5t\" (UID: \"65e086a0-1eb5-46cf-895e-8a9cd892ce6b\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.483938 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.483837    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e086a0-1eb5-46cf-895e-8a9cd892ce6b-tls-certs\") pod \"model-serving-api-75bd89fdfc-gmb5t\" (UID: \"65e086a0-1eb5-46cf-895e-8a9cd892ce6b\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.584996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.584968    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c50d69b-8557-482a-8329-976d4556bf95-cert\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.585182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.585041    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5l6c\" (UniqueName: \"kubernetes.io/projected/65e086a0-1eb5-46cf-895e-8a9cd892ce6b-kube-api-access-w5l6c\") pod \"model-serving-api-75bd89fdfc-gmb5t\" (UID: \"65e086a0-1eb5-46cf-895e-8a9cd892ce6b\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.585182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.585127    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e086a0-1eb5-46cf-895e-8a9cd892ce6b-tls-certs\") pod \"model-serving-api-75bd89fdfc-gmb5t\" (UID: \"65e086a0-1eb5-46cf-895e-8a9cd892ce6b\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.585182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.585172    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnr6n\" (UniqueName: \"kubernetes.io/projected/3c50d69b-8557-482a-8329-976d4556bf95-kube-api-access-jnr6n\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.585336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.585238    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c50d69b-8557-482a-8329-976d4556bf95-metrics-certs\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.587611 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.587576    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e086a0-1eb5-46cf-895e-8a9cd892ce6b-tls-certs\") pod \"model-serving-api-75bd89fdfc-gmb5t\" (UID: \"65e086a0-1eb5-46cf-895e-8a9cd892ce6b\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.595407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.595350    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5l6c\" (UniqueName: \"kubernetes.io/projected/65e086a0-1eb5-46cf-895e-8a9cd892ce6b-kube-api-access-w5l6c\") pod \"model-serving-api-75bd89fdfc-gmb5t\" (UID: \"65e086a0-1eb5-46cf-895e-8a9cd892ce6b\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.660685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.660652    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:20.685844 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.685819    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnr6n\" (UniqueName: \"kubernetes.io/projected/3c50d69b-8557-482a-8329-976d4556bf95-kube-api-access-jnr6n\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.685968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.685879    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c50d69b-8557-482a-8329-976d4556bf95-metrics-certs\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.685968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.685912    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c50d69b-8557-482a-8329-976d4556bf95-cert\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.688528 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.688502    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c50d69b-8557-482a-8329-976d4556bf95-metrics-certs\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.688654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.688535    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c50d69b-8557-482a-8329-976d4556bf95-cert\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.694060 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.694031    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnr6n\" (UniqueName: \"kubernetes.io/projected/3c50d69b-8557-482a-8329-976d4556bf95-kube-api-access-jnr6n\") pod \"odh-model-controller-769875db56-nmdfz\" (UID: \"3c50d69b-8557-482a-8329-976d4556bf95\") " pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.748175 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.748148    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:20.945467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:20.945438    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gmb5t"]
Oct 31 01:05:20.950237 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:05:20.950203    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e086a0_1eb5_46cf_895e_8a9cd892ce6b.slice/crio-7eca2e292df91ac95f2865271f288421d9c2d3f62a594f4adfd3ee290875e494 WatchSource:0}: Error finding container 7eca2e292df91ac95f2865271f288421d9c2d3f62a594f4adfd3ee290875e494: Status 404 returned error can't find the container with id 7eca2e292df91ac95f2865271f288421d9c2d3f62a594f4adfd3ee290875e494
Oct 31 01:05:21.023991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:21.023953    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-nmdfz"]
Oct 31 01:05:21.027233 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:05:21.027204    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c50d69b_8557_482a_8329_976d4556bf95.slice/crio-249943880db56ac2c8da17fbeae1dd7d5b5c0c8013b0a76c66a49a40bc5e6a38 WatchSource:0}: Error finding container 249943880db56ac2c8da17fbeae1dd7d5b5c0c8013b0a76c66a49a40bc5e6a38: Status 404 returned error can't find the container with id 249943880db56ac2c8da17fbeae1dd7d5b5c0c8013b0a76c66a49a40bc5e6a38
Oct 31 01:05:21.256899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:21.256815    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-nmdfz" event={"ID":"3c50d69b-8557-482a-8329-976d4556bf95","Type":"ContainerStarted","Data":"249943880db56ac2c8da17fbeae1dd7d5b5c0c8013b0a76c66a49a40bc5e6a38"}
Oct 31 01:05:21.257954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:21.257928    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t" event={"ID":"65e086a0-1eb5-46cf-895e-8a9cd892ce6b","Type":"ContainerStarted","Data":"7eca2e292df91ac95f2865271f288421d9c2d3f62a594f4adfd3ee290875e494"}
Oct 31 01:05:23.271961 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:23.271927    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t" event={"ID":"65e086a0-1eb5-46cf-895e-8a9cd892ce6b","Type":"ContainerStarted","Data":"c9b89afa8b04b7215cab408bdb2ce8914cefe10d9560147e38a686da7e42e259"}
Oct 31 01:05:23.272401 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:23.272006    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:24.278773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:24.278654    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-nmdfz" event={"ID":"3c50d69b-8557-482a-8329-976d4556bf95","Type":"ContainerStarted","Data":"aba1b443b47227c286129bc6dab313c6b7cdfd022f51c818df64883031d9ee40"}
Oct 31 01:05:24.296665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:24.296618    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-nmdfz" podStartSLOduration=1.395841045 podStartE2EDuration="4.296587306s" totalImagesPullingTime="2.900746261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:05:24.29537979 +0000 UTC m=+521.282491767" watchObservedRunningTime="2026-10-31 01:05:24.296587306 +0000 UTC m=+521.283699334"
Oct 31 01:05:24.296946 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:24.296912    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t" podStartSLOduration=2.20505381 podStartE2EDuration="4.29690664s" totalImagesPullingTime="2.09185283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:05:23.288689117 +0000 UTC m=+520.275801092" watchObservedRunningTime="2026-10-31 01:05:24.29690664 +0000 UTC m=+521.284018631"
Oct 31 01:05:25.283411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:25.283379    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:34.283336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:34.283305    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-gmb5t"
Oct 31 01:05:36.292370 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:36.292338    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-nmdfz"
Oct 31 01:05:49.442346 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:05:49.442242    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:05:57.701913 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.701876    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-wckbc"]
Oct 31 01:05:57.706011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.705986    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:57.713644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.712990    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-wckbc"]
Oct 31 01:05:57.808184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.808146    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r26c\" (UniqueName: \"kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c\") pod \"authorino-6f6944bcdd-wckbc\" (UID: \"34772a7a-b945-45ce-ab6a-62c52c45baac\") " pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:57.881621 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.881567    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-wckbc"]
Oct 31 01:05:57.881812 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:05:57.881792    8029 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2r26c], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-wckbc" podUID="34772a7a-b945-45ce-ab6a-62c52c45baac"
Oct 31 01:05:57.908408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.908374    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ltgjf"]
Oct 31 01:05:57.908710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.908689    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r26c\" (UniqueName: \"kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c\") pod \"authorino-6f6944bcdd-wckbc\" (UID: \"34772a7a-b945-45ce-ab6a-62c52c45baac\") " pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:57.913838 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.913816    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:57.919502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.919442    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 31 01:05:57.921219 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.921174    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ltgjf"]
Oct 31 01:05:57.923517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:57.923491    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r26c\" (UniqueName: \"kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c\") pod \"authorino-6f6944bcdd-wckbc\" (UID: \"34772a7a-b945-45ce-ab6a-62c52c45baac\") " pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:58.009536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.009453    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert\") pod \"authorino-6889cbd8c4-ltgjf\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") " pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.009536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.009495    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4x5j\" (UniqueName: \"kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j\") pod \"authorino-6889cbd8c4-ltgjf\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") " pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.066624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.066568    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ltgjf"]
Oct 31 01:05:58.066869 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:05:58.066846    8029 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-n4x5j tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-ltgjf" podUID="f4c1a905-84fc-4d22-8448-21d48b56119d"
Oct 31 01:05:58.110867 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.110834    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert\") pod \"authorino-6889cbd8c4-ltgjf\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") " pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.111034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.110875    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4x5j\" (UniqueName: \"kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j\") pod \"authorino-6889cbd8c4-ltgjf\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") " pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.113357 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.113334    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert\") pod \"authorino-6889cbd8c4-ltgjf\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") " pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.119048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.119020    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4x5j\" (UniqueName: \"kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j\") pod \"authorino-6889cbd8c4-ltgjf\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") " pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.438675 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.438650    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.438856 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.438650    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:58.474971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.474947    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:58.490753 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.490720    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:58.615326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.615294    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert\" (UniqueName: \"kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert\") pod \"f4c1a905-84fc-4d22-8448-21d48b56119d\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") "
Oct 31 01:05:58.615492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.615365    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j\" (UniqueName: \"kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j\") pod \"f4c1a905-84fc-4d22-8448-21d48b56119d\" (UID: \"f4c1a905-84fc-4d22-8448-21d48b56119d\") "
Oct 31 01:05:58.615492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.615413    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c\" (UniqueName: \"kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c\") pod \"34772a7a-b945-45ce-ab6a-62c52c45baac\" (UID: \"34772a7a-b945-45ce-ab6a-62c52c45baac\") "
Oct 31 01:05:58.617543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.617508    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert" pod "f4c1a905-84fc-4d22-8448-21d48b56119d" (UID: "f4c1a905-84fc-4d22-8448-21d48b56119d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:05:58.617688 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.617558    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c" pod "34772a7a-b945-45ce-ab6a-62c52c45baac" (UID: "34772a7a-b945-45ce-ab6a-62c52c45baac"). InnerVolumeSpecName "kube-api-access-2r26c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:05:58.617688 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.617564    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j" pod "f4c1a905-84fc-4d22-8448-21d48b56119d" (UID: "f4c1a905-84fc-4d22-8448-21d48b56119d"). InnerVolumeSpecName "kube-api-access-n4x5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:05:58.716931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.716850    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r26c\" (UniqueName: \"kubernetes.io/projected/34772a7a-b945-45ce-ab6a-62c52c45baac-kube-api-access-2r26c\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:58.716931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.716880    8029 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f4c1a905-84fc-4d22-8448-21d48b56119d-tls-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:58.716931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:58.716890    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4x5j\" (UniqueName: \"kubernetes.io/projected/f4c1a905-84fc-4d22-8448-21d48b56119d-kube-api-access-n4x5j\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:05:59.442514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:59.442434    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-wckbc"
Oct 31 01:05:59.442679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:59.442439    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ltgjf"
Oct 31 01:05:59.476491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:59.476464    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-wckbc"]
Oct 31 01:05:59.483088 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:59.483062    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-wckbc"]
Oct 31 01:05:59.508475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:59.508449    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ltgjf"]
Oct 31 01:05:59.512802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:05:59.512749    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ltgjf"]
Oct 31 01:06:00.959882 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:00.959851    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hh27j"]
Oct 31 01:06:00.960394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:00.960037    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j" podUID="d8f360ff-3c50-4dc5-af9e-19058b40b72b" containerName="authorino" containerID="cri-o://6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f" gracePeriod=30
Oct 31 01:06:01.181658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.181628    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34772a7a-b945-45ce-ab6a-62c52c45baac" path="/var/lib/kubelet/pods/34772a7a-b945-45ce-ab6a-62c52c45baac/volumes"
Oct 31 01:06:01.181900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.181887    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4c1a905-84fc-4d22-8448-21d48b56119d" path="/var/lib/kubelet/pods/f4c1a905-84fc-4d22-8448-21d48b56119d/volumes"
Oct 31 01:06:01.206093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.206072    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:06:01.341374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.341295    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j\" (UniqueName: \"kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j\") pod \"d8f360ff-3c50-4dc5-af9e-19058b40b72b\" (UID: \"d8f360ff-3c50-4dc5-af9e-19058b40b72b\") "
Oct 31 01:06:01.343462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.343439    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j" pod "d8f360ff-3c50-4dc5-af9e-19058b40b72b" (UID: "d8f360ff-3c50-4dc5-af9e-19058b40b72b"). InnerVolumeSpecName "kube-api-access-9kb9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:06:01.442082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.442054    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kb9j\" (UniqueName: \"kubernetes.io/projected/d8f360ff-3c50-4dc5-af9e-19058b40b72b-kube-api-access-9kb9j\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:06:01.452766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.452732    8029 generic.go:356] "Generic (PLEG): container finished" podID="d8f360ff-3c50-4dc5-af9e-19058b40b72b" containerID="6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f" exitCode=0
Oct 31 01:06:01.452883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.452780    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j"
Oct 31 01:06:01.452883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.452811    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j" event={"ID":"d8f360ff-3c50-4dc5-af9e-19058b40b72b","Type":"ContainerDied","Data":"6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f"}
Oct 31 01:06:01.452883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.452848    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hh27j" event={"ID":"d8f360ff-3c50-4dc5-af9e-19058b40b72b","Type":"ContainerDied","Data":"d57cb8358bc6e8c2d68d516d9c04a77647cfe04f105f7133ef3c28f6ddea2381"}
Oct 31 01:06:01.452883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.452864    8029 scope.go:122] "RemoveContainer" containerID="6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f"
Oct 31 01:06:01.461693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.461677    8029 scope.go:122] "RemoveContainer" containerID="6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f"
Oct 31 01:06:01.461965 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:06:01.461935    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f\": container with ID starting with 6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f not found: ID does not exist" containerID="6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f"
Oct 31 01:06:01.462048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.461977    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f"} err="failed to get container status \"6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f\": rpc error: code = NotFound desc = could not find container \"6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f\": container with ID starting with 6b4f5a88069f971b569bf1aa1e562d517331fcbcbc4e29210181aa987b3f243f not found: ID does not exist"
Oct 31 01:06:01.472450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.472426    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hh27j"]
Oct 31 01:06:01.480931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:01.480904    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hh27j"]
Oct 31 01:06:03.181884 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:03.181853    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8f360ff-3c50-4dc5-af9e-19058b40b72b" path="/var/lib/kubelet/pods/d8f360ff-3c50-4dc5-af9e-19058b40b72b/volumes"
Oct 31 01:06:19.441553 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:06:19.441506    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:06:43.224111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:43.224083    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:06:43.232190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:06:43.232171    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:06:49.440980 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:06:49.440933    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:07:14.968686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.968651    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"]
Oct 31 01:07:14.969148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.969049    8029 cpu_manager.go:414] "containerMap: removing container" podUID="d8f360ff-3c50-4dc5-af9e-19058b40b72b" containerName="authorino"
Oct 31 01:07:14.969148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.969061    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8f360ff-3c50-4dc5-af9e-19058b40b72b" containerName="authorino"
Oct 31 01:07:14.969148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.969140    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8f360ff-3c50-4dc5-af9e-19058b40b72b" containerName="authorino"
Oct 31 01:07:14.980071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.980046    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:14.982356 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.982330    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"]
Oct 31 01:07:14.984959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.984838    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 01:07:14.984959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.984902    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 01:07:14.984959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.984935    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 31 01:07:14.985153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:14.985032    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8kl5l\""
Oct 31 01:07:15.084227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.084192    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.084383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.084251    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.084383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.084332    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wnl2\" (UniqueName: \"kubernetes.io/projected/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-kube-api-access-2wnl2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.084461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.084409    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.084461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.084448    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.084523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.084470    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185345    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185380    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185407    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185454    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185473    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wnl2\" (UniqueName: \"kubernetes.io/projected/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-kube-api-access-2wnl2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185511    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185802    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185868 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185835    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.185962 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.185943    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.187898 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.187874    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.188045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.188029    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.193336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.193319    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wnl2\" (UniqueName: \"kubernetes.io/projected/4ad09d17-5b0f-4b20-9e81-7c0faf104aae-kube-api-access-2wnl2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq\" (UID: \"4ad09d17-5b0f-4b20-9e81-7c0faf104aae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.294665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.294580    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:15.576060 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.575966    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"]
Oct 31 01:07:15.580123 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:07:15.580100    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad09d17_5b0f_4b20_9e81_7c0faf104aae.slice/crio-3d92bbdf6e1c1fdcf68a5082f96818406356bfd8a4fc379c88ca3c6028d2965e WatchSource:0}: Error finding container 3d92bbdf6e1c1fdcf68a5082f96818406356bfd8a4fc379c88ca3c6028d2965e: Status 404 returned error can't find the container with id 3d92bbdf6e1c1fdcf68a5082f96818406356bfd8a4fc379c88ca3c6028d2965e
Oct 31 01:07:15.776310 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:15.776276    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq" event={"ID":"4ad09d17-5b0f-4b20-9e81-7c0faf104aae","Type":"ContainerStarted","Data":"3d92bbdf6e1c1fdcf68a5082f96818406356bfd8a4fc379c88ca3c6028d2965e"}
Oct 31 01:07:17.150517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.150481    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"]
Oct 31 01:07:17.154423 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.154402    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.161115 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.161090    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 31 01:07:17.168429 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.168407    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"]
Oct 31 01:07:17.202691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.202667    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.202770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.202707    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.202770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.202732    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.202921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.202898    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.202980 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.202961    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0208e85e-36ac-4a08-be16-15738c866d3e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.203046 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.203029    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6lss\" (UniqueName: \"kubernetes.io/projected/0208e85e-36ac-4a08-be16-15738c866d3e-kube-api-access-q6lss\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304047    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304099    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0208e85e-36ac-4a08-be16-15738c866d3e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304300 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304287    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6lss\" (UniqueName: \"kubernetes.io/projected/0208e85e-36ac-4a08-be16-15738c866d3e-kube-api-access-q6lss\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304356 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304343    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304401 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304382    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304414    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304477    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304724    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.304877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.304749    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.306655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.306636    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0208e85e-36ac-4a08-be16-15738c866d3e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.307038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.307020    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0208e85e-36ac-4a08-be16-15738c866d3e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.312299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.312277    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6lss\" (UniqueName: \"kubernetes.io/projected/0208e85e-36ac-4a08-be16-15738c866d3e-kube-api-access-q6lss\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g\" (UID: \"0208e85e-36ac-4a08-be16-15738c866d3e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.467635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.467523    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:17.741132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.741063    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"]
Oct 31 01:07:17.744208 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:07:17.744183    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0208e85e_36ac_4a08_be16_15738c866d3e.slice/crio-17eaa2682270216b8070e1739e33da589b3ddfe857b55a14cd3861a110c6dd93 WatchSource:0}: Error finding container 17eaa2682270216b8070e1739e33da589b3ddfe857b55a14cd3861a110c6dd93: Status 404 returned error can't find the container with id 17eaa2682270216b8070e1739e33da589b3ddfe857b55a14cd3861a110c6dd93
Oct 31 01:07:17.787387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.787363    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g" event={"ID":"0208e85e-36ac-4a08-be16-15738c866d3e","Type":"ContainerStarted","Data":"17eaa2682270216b8070e1739e33da589b3ddfe857b55a14cd3861a110c6dd93"}
Oct 31 01:07:17.788792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.788765    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq" event={"ID":"4ad09d17-5b0f-4b20-9e81-7c0faf104aae","Type":"ContainerStarted","Data":"3769b99173a7861859ee05833e2cb9a3f69b45fb3a60d766ba2fee47269e97ff"}
Oct 31 01:07:17.805416 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:17.805373    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq" podStartSLOduration=2.208071951 podStartE2EDuration="3.805361447s" totalImagesPullingTime="1.597289496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:17.804018064 +0000 UTC m=+634.791130040" watchObservedRunningTime="2026-10-31 01:07:17.805361447 +0000 UTC m=+634.792473475"
Oct 31 01:07:18.295076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:18.295045    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:18.794591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:18.794506    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g" event={"ID":"0208e85e-36ac-4a08-be16-15738c866d3e","Type":"ContainerStarted","Data":"ff4bed9c95d2ff78b3e0fd708c88dffc6abeab206be751c07de4877b4de96c5a"}
Oct 31 01:07:18.812094 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:18.812048    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g" podStartSLOduration=1.497315955 podStartE2EDuration="1.81203438s" totalImagesPullingTime="314.718425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:18.810403868 +0000 UTC m=+635.797515845" watchObservedRunningTime="2026-10-31 01:07:18.81203438 +0000 UTC m=+635.799146366"
Oct 31 01:07:19.442344 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:07:19.442311    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:07:19.468581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.468550    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:19.514553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.514484    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"]
Oct 31 01:07:19.518484 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.518463    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.527438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.527297    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 31 01:07:19.528445 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.528384    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"]
Oct 31 01:07:19.625749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.625717    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.625902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.625769    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.625902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.625784    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.625902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.625832    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mfwj\" (UniqueName: \"kubernetes.io/projected/f13faa59-6cd2-4a45-818a-bba8f0daaaad-kube-api-access-6mfwj\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.625902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.625893    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.626036 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.625965    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13faa59-6cd2-4a45-818a-bba8f0daaaad-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.726843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.726776    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.726843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.726826    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13faa59-6cd2-4a45-818a-bba8f0daaaad-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727010 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.726874    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.727020    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.727061    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727142 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.727086    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfwj\" (UniqueName: \"kubernetes.io/projected/f13faa59-6cd2-4a45-818a-bba8f0daaaad-kube-api-access-6mfwj\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727247 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.727225    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.727344    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.727429 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.727382    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.729442 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.729420    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13faa59-6cd2-4a45-818a-bba8f0daaaad-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.729543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.729496    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13faa59-6cd2-4a45-818a-bba8f0daaaad-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.734418 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.734400    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfwj\" (UniqueName: \"kubernetes.io/projected/f13faa59-6cd2-4a45-818a-bba8f0daaaad-kube-api-access-6mfwj\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz\" (UID: \"f13faa59-6cd2-4a45-818a-bba8f0daaaad\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:19.833994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:19.833964    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:20.106272 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:20.106180    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"]
Oct 31 01:07:20.109003 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:07:20.108977    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13faa59_6cd2_4a45_818a_bba8f0daaaad.slice/crio-e03836fb8a244cdf7aacf156728b2b0fdfaa41ddf569c31cd5d6ae64a4a7a290 WatchSource:0}: Error finding container e03836fb8a244cdf7aacf156728b2b0fdfaa41ddf569c31cd5d6ae64a4a7a290: Status 404 returned error can't find the container with id e03836fb8a244cdf7aacf156728b2b0fdfaa41ddf569c31cd5d6ae64a4a7a290
Oct 31 01:07:20.804145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:20.804050    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz" event={"ID":"f13faa59-6cd2-4a45-818a-bba8f0daaaad","Type":"ContainerStarted","Data":"9a89cc049f05ec30e6510c1043f11a169192638d14d7e53aac738df47388f15f"}
Oct 31 01:07:20.804145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:20.804112    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz" event={"ID":"f13faa59-6cd2-4a45-818a-bba8f0daaaad","Type":"ContainerStarted","Data":"e03836fb8a244cdf7aacf156728b2b0fdfaa41ddf569c31cd5d6ae64a4a7a290"}
Oct 31 01:07:20.821282 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:20.821222    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz" podStartSLOduration=1.494351902 podStartE2EDuration="1.821208059s" totalImagesPullingTime="326.856157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:20.819997197 +0000 UTC m=+637.807109172" watchObservedRunningTime="2026-10-31 01:07:20.821208059 +0000 UTC m=+637.808320034"
Oct 31 01:07:20.835065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:20.835026    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:25.295622 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:25.295571    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:25.308245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:25.308219    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:25.838297 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:25.838269    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq"
Oct 31 01:07:26.023505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.023473    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"]
Oct 31 01:07:26.027114 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.027092    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.030637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.030616    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 31 01:07:26.038214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.036509    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"]
Oct 31 01:07:26.084148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.084113    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.084314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.084158    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.084314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.084180    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzlds\" (UniqueName: \"kubernetes.io/projected/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-kube-api-access-vzlds\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.084443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.084355    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.084443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.084409    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.084513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.084471    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.185752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.185717    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.185752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.185748    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzlds\" (UniqueName: \"kubernetes.io/projected/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-kube-api-access-vzlds\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.185978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.185784    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.185978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.185829    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.185978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.185869    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.186145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.186024    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.186198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.186135    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.186257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.186218    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.186330 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.186308    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.188040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.188024    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.188386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.188366    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.194497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.194478    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzlds\" (UniqueName: \"kubernetes.io/projected/78f6922d-f4aa-453c-a58a-7bb0f6cc496c-kube-api-access-vzlds\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bqpc7\" (UID: \"78f6922d-f4aa-453c-a58a-7bb0f6cc496c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.342260 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.342233    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:26.622673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.617878    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"]
Oct 31 01:07:26.625990 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:07:26.625964    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f6922d_f4aa_453c_a58a_7bb0f6cc496c.slice/crio-e081aa9bd79430543463526394bb8603bcbefbe1915fb35c14b89ec0620ede0c WatchSource:0}: Error finding container e081aa9bd79430543463526394bb8603bcbefbe1915fb35c14b89ec0620ede0c: Status 404 returned error can't find the container with id e081aa9bd79430543463526394bb8603bcbefbe1915fb35c14b89ec0620ede0c
Oct 31 01:07:26.832967 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:26.832924    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7" event={"ID":"78f6922d-f4aa-453c-a58a-7bb0f6cc496c","Type":"ContainerStarted","Data":"e081aa9bd79430543463526394bb8603bcbefbe1915fb35c14b89ec0620ede0c"}
Oct 31 01:07:27.467768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:27.467736    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:27.480690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:27.480671    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:27.839423 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:27.839342    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7" event={"ID":"78f6922d-f4aa-453c-a58a-7bb0f6cc496c","Type":"ContainerStarted","Data":"ae77a377bdc048b3071f1c8b9f8f59a1c640e6ab3ebe36a916505bf44c10d74c"}
Oct 31 01:07:27.850747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:27.850725    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g"
Oct 31 01:07:27.880931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:27.880885    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7" podStartSLOduration=1.581066297 podStartE2EDuration="1.8808717s" totalImagesPullingTime="299.805403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:27.88035556 +0000 UTC m=+644.867467536" watchObservedRunningTime="2026-10-31 01:07:27.8808717 +0000 UTC m=+644.867983673"
Oct 31 01:07:28.342819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:28.342786    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:29.834157 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:29.834129    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:29.847141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:29.847119    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:29.861444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:29.861424    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz"
Oct 31 01:07:36.342439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.342403    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:36.354637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.354591    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:36.542774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.542743    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"]
Oct 31 01:07:36.546825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.546810    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.552396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.552374    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 01:07:36.552552 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.552397    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nj7zm\""
Oct 31 01:07:36.557413 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.557364    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"]
Oct 31 01:07:36.678107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678079    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678122    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc97d91-1e29-430f-b333-61b323418290-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678140    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5vqm\" (UniqueName: \"kubernetes.io/projected/5bc97d91-1e29-430f-b333-61b323418290-kube-api-access-h5vqm\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678340 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678262    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678340 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678300    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678403 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678366    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678402    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc97d91-1e29-430f-b333-61b323418290-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678425    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.678498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.678485    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc97d91-1e29-430f-b333-61b323418290-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779538    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779573    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779623    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc97d91-1e29-430f-b333-61b323418290-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779650    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779695    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc97d91-1e29-430f-b333-61b323418290-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779760    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779802    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc97d91-1e29-430f-b333-61b323418290-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779825    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5vqm\" (UniqueName: \"kubernetes.io/projected/5bc97d91-1e29-430f-b333-61b323418290-kube-api-access-h5vqm\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.779994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.779882    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.780133 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.780027    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.780133 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.780106    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.780400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.780373    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.780513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.780397    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.780513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.780398    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc97d91-1e29-430f-b333-61b323418290-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.782332 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.782311    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc97d91-1e29-430f-b333-61b323418290-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.782530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.782511    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc97d91-1e29-430f-b333-61b323418290-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.787883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.787864    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc97d91-1e29-430f-b333-61b323418290-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.788046 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.788029    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5vqm\" (UniqueName: \"kubernetes.io/projected/5bc97d91-1e29-430f-b333-61b323418290-kube-api-access-h5vqm\") pod \"maas-default-gateway-openshift-default-855f9574cf-tw6hh\" (UID: \"5bc97d91-1e29-430f-b333-61b323418290\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.862901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.862878    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:36.895101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:36.895080    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bqpc7"
Oct 31 01:07:37.142721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:37.142686    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"]
Oct 31 01:07:37.888239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:37.888206    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" event={"ID":"5bc97d91-1e29-430f-b333-61b323418290","Type":"ContainerStarted","Data":"094e9259450d7016b71ef2c581182757260e19765b0edc87f593b696d9ffeff9"}
Oct 31 01:07:40.435086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:40.435053    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:40.435347 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:40.435127    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:40.435347 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:40.435157    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:40.908391 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:40.908352    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" event={"ID":"5bc97d91-1e29-430f-b333-61b323418290","Type":"ContainerStarted","Data":"696a8d1b5b05c1043964358e8685f4d7ad68d20642e1a5a5f2eee309f084ef4f"}
Oct 31 01:07:40.929219 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:40.929167    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podStartSLOduration=1.642436995 podStartE2EDuration="4.929152404s" totalImagesPullingTime="3.286715409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:40.927670786 +0000 UTC m=+657.914782775" watchObservedRunningTime="2026-10-31 01:07:40.929152404 +0000 UTC m=+657.916264381"
Oct 31 01:07:41.863830 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:41.863793    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:41.865230 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:41.865202    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 01:07:41.865357 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:41.865259    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 01:07:42.863301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:42.863267    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 01:07:42.863491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:42.863339    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 01:07:43.863301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:43.863263    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 01:07:43.863711 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:43.863313    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 01:07:44.863916 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:44.863885    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 01:07:44.864396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:44.863942    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 01:07:45.863472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:45.863427    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 01:07:45.863771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:45.863499    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 01:07:46.864132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:46.864103    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 01:07:46.864529 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:46.864157    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 01:07:48.569739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:48.569708    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:48.570183 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:48.570054    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:48.571111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:48.571088    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh"
Oct 31 01:07:49.441949 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:07:49.441905    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:07:51.565339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.565306    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"]
Oct 31 01:07:51.569712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.569691    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.581304 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.581279    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"]
Oct 31 01:07:51.598931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.598908    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"]
Oct 31 01:07:51.603739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.603717    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.613814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.613753    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"]
Oct 31 01:07:51.727150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727117    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727154    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727173    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727190    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727332    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727363    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndmrz\" (UniqueName: \"kubernetes.io/projected/38590073-ac40-4f8e-8295-5bc08ebc313b-kube-api-access-ndmrz\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727389    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727460    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727532 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727503    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727568 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727539    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38590073-ac40-4f8e-8295-5bc08ebc313b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727577    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727656    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727691    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727713    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-926zf\" (UniqueName: \"kubernetes.io/projected/d6fd01e9-d8ad-4360-8648-e90ff85643b7-kube-api-access-926zf\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727730    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727798 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727778    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.727830 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727797    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.727830 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.727824    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828292    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828327    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828348    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-926zf\" (UniqueName: \"kubernetes.io/projected/d6fd01e9-d8ad-4360-8648-e90ff85643b7-kube-api-access-926zf\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828364    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828382    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828405    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828444    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828479    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828514    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828537    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828563    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828638    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828670    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndmrz\" (UniqueName: \"kubernetes.io/projected/38590073-ac40-4f8e-8295-5bc08ebc313b-kube-api-access-ndmrz\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828703    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828730    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828745    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.828765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828759    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.829337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828814    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38590073-ac40-4f8e-8295-5bc08ebc313b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.829337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.828868    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.829337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829090    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.829337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829209    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.829337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829274    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.829337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829289    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.829661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829354    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.829746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829726    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.829942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.829921    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.830150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.830126    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38590073-ac40-4f8e-8295-5bc08ebc313b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.830285 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.830259    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.831311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.831282    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.831839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.831814    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.832152 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.832131    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.832337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.832321    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.836533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.836511    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-926zf\" (UniqueName: \"kubernetes.io/projected/d6fd01e9-d8ad-4360-8648-e90ff85643b7-kube-api-access-926zf\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.836766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.836748    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndmrz\" (UniqueName: \"kubernetes.io/projected/38590073-ac40-4f8e-8295-5bc08ebc313b-kube-api-access-ndmrz\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.837066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.837047    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38590073-ac40-4f8e-8295-5bc08ebc313b-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-sds66\" (UID: \"38590073-ac40-4f8e-8295-5bc08ebc313b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:51.837341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.837324    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6fd01e9-d8ad-4360-8648-e90ff85643b7-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-qlmqk\" (UID: \"d6fd01e9-d8ad-4360-8648-e90ff85643b7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.884917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.884894    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:51.919320 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:51.919295    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:52.164658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.164636    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"]
Oct 31 01:07:52.167173 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:07:52.167144    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fd01e9_d8ad_4360_8648_e90ff85643b7.slice/crio-a1d2b2f498015257bdfa9b995e1ecb21b56e197aa1f82676f17f9b4c617561c2 WatchSource:0}: Error finding container a1d2b2f498015257bdfa9b995e1ecb21b56e197aa1f82676f17f9b4c617561c2: Status 404 returned error can't find the container with id a1d2b2f498015257bdfa9b995e1ecb21b56e197aa1f82676f17f9b4c617561c2
Oct 31 01:07:52.169431 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.169401    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:52.169489 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.169467    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:52.169523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.169494    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:52.207540 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.207516    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"]
Oct 31 01:07:52.210749 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:07:52.210724    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38590073_ac40_4f8e_8295_5bc08ebc313b.slice/crio-e513a652229d4bb94f65d194a7f8bf0b0d9667356e9538ba0e6a29066bfd8fa8 WatchSource:0}: Error finding container e513a652229d4bb94f65d194a7f8bf0b0d9667356e9538ba0e6a29066bfd8fa8: Status 404 returned error can't find the container with id e513a652229d4bb94f65d194a7f8bf0b0d9667356e9538ba0e6a29066bfd8fa8
Oct 31 01:07:52.213160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.213135    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:52.213236 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.213190    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:52.213236 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.213214    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:07:52.967877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.967771    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" event={"ID":"38590073-ac40-4f8e-8295-5bc08ebc313b","Type":"ContainerStarted","Data":"6e7bb17cf1ba1f41f0193bdc631dcc143749470078c9d384be088c955f56cb42"}
Oct 31 01:07:52.967877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.967860    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" event={"ID":"38590073-ac40-4f8e-8295-5bc08ebc313b","Type":"ContainerStarted","Data":"e513a652229d4bb94f65d194a7f8bf0b0d9667356e9538ba0e6a29066bfd8fa8"}
Oct 31 01:07:52.969287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.969262    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" event={"ID":"d6fd01e9-d8ad-4360-8648-e90ff85643b7","Type":"ContainerStarted","Data":"684c78eec47afeb4fccb86786d7d74441595a3558f036d26bdced12adf544074"}
Oct 31 01:07:52.969426 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.969301    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" event={"ID":"d6fd01e9-d8ad-4360-8648-e90ff85643b7","Type":"ContainerStarted","Data":"a1d2b2f498015257bdfa9b995e1ecb21b56e197aa1f82676f17f9b4c617561c2"}
Oct 31 01:07:53.014628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:52.995265    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podStartSLOduration=1.9952463169999999 podStartE2EDuration="1.995246317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:52.993069719 +0000 UTC m=+669.980181696" watchObservedRunningTime="2026-10-31 01:07:52.995246317 +0000 UTC m=+669.982358298"
Oct 31 01:07:53.023627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.020857    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podStartSLOduration=2.020807019 podStartE2EDuration="2.020807019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:07:53.020116883 +0000 UTC m=+670.007228860" watchObservedRunningTime="2026-10-31 01:07:53.020807019 +0000 UTC m=+670.007918999"
Oct 31 01:07:53.895628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.893704    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:07:53.903625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.901726    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-qlmqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 01:07:53.903625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.901801    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podUID="d6fd01e9-d8ad-4360-8648-e90ff85643b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 01:07:53.923629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.921798    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:07:53.930634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.929726    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-sds66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 01:07:53.930634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:53.929780    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podUID="38590073-ac40-4f8e-8295-5bc08ebc313b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 01:07:54.905638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:54.901707    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-qlmqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 01:07:54.905638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:54.901769    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podUID="d6fd01e9-d8ad-4360-8648-e90ff85643b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 01:07:54.925635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:54.923807    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-sds66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 01:07:54.925635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:54.923864    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podUID="38590073-ac40-4f8e-8295-5bc08ebc313b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 01:07:55.885903 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:55.885872    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-qlmqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 01:07:55.886176 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:55.886149    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podUID="d6fd01e9-d8ad-4360-8648-e90ff85643b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 01:07:55.923629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:55.921093    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-sds66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 01:07:55.923629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:55.921146    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podUID="38590073-ac40-4f8e-8295-5bc08ebc313b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 01:07:56.886980 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:56.886946    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-qlmqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 01:07:56.887303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:56.887251    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podUID="d6fd01e9-d8ad-4360-8648-e90ff85643b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 01:07:56.928627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:56.927698    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-sds66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 01:07:56.928627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:56.927752    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podUID="38590073-ac40-4f8e-8295-5bc08ebc313b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 01:07:57.885629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:57.885113    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-qlmqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 01:07:57.885629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:57.885175    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podUID="d6fd01e9-d8ad-4360-8648-e90ff85643b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 01:07:57.922628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:57.920096    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-sds66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 01:07:57.922628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:57.920153    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podUID="38590073-ac40-4f8e-8295-5bc08ebc313b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 01:07:58.885415 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:58.885379    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-qlmqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 01:07:58.885846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:58.885444    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk" podUID="d6fd01e9-d8ad-4360-8648-e90ff85643b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 01:07:58.920710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:58.920670    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-sds66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 01:07:58.920916 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:07:58.920723    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66" podUID="38590073-ac40-4f8e-8295-5bc08ebc313b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 01:08:00.430067 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:00.430038    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:08:00.430577 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:00.430553    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:08:00.431680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:00.431656    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-sds66"
Oct 31 01:08:00.444615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:00.444579    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:08:00.444910 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:00.444891    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:08:00.445565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:00.445543    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-qlmqk"
Oct 31 01:08:03.295386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.295353    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-669748c564-fn8th"]
Oct 31 01:08:03.299402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.299384    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.306088 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.306061    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-669748c564-fn8th"]
Oct 31 01:08:03.308290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.308268    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9t9bz\""
Oct 31 01:08:03.308431 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.308417    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 31 01:08:03.350823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.350794    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 01:08:03.350975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.350870    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 01:08:03.386430 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.386399    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b-tls-cert\") pod \"authorino-669748c564-fn8th\" (UID: \"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b\") " pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.386561 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.386432    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtcmn\" (UniqueName: \"kubernetes.io/projected/83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b-kube-api-access-xtcmn\") pod \"authorino-669748c564-fn8th\" (UID: \"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b\") " pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.487113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.487081    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b-tls-cert\") pod \"authorino-669748c564-fn8th\" (UID: \"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b\") " pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.487113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.487112    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtcmn\" (UniqueName: \"kubernetes.io/projected/83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b-kube-api-access-xtcmn\") pod \"authorino-669748c564-fn8th\" (UID: \"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b\") " pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.489609 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.489581    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b-tls-cert\") pod \"authorino-669748c564-fn8th\" (UID: \"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b\") " pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.495217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.495199    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtcmn\" (UniqueName: \"kubernetes.io/projected/83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b-kube-api-access-xtcmn\") pod \"authorino-669748c564-fn8th\" (UID: \"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b\") " pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.613574 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.613545    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669748c564-fn8th"
Oct 31 01:08:03.885060 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:03.885037    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-669748c564-fn8th"]
Oct 31 01:08:03.888366 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:08:03.888336    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83fe9f2a_bb1d_4f8b_9e1a_dce0ff96204b.slice/crio-9982e4ccc00c3c00f513acaf97a6a746d24a9ca3cf76824e6de2c7a95a611491 WatchSource:0}: Error finding container 9982e4ccc00c3c00f513acaf97a6a746d24a9ca3cf76824e6de2c7a95a611491: Status 404 returned error can't find the container with id 9982e4ccc00c3c00f513acaf97a6a746d24a9ca3cf76824e6de2c7a95a611491
Oct 31 01:08:04.067255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:04.067221    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669748c564-fn8th" event={"ID":"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b","Type":"ContainerStarted","Data":"9982e4ccc00c3c00f513acaf97a6a746d24a9ca3cf76824e6de2c7a95a611491"}
Oct 31 01:08:06.096626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:06.096402    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669748c564-fn8th" event={"ID":"83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b","Type":"ContainerStarted","Data":"ee8f519433169a4f445e923da0b4098fc57d3e982a798a8f832d995a3b860b20"}
Oct 31 01:08:06.116626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:08:06.115582    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-669748c564-fn8th" podStartSLOduration=1.968945824 podStartE2EDuration="3.115565549s" totalImagesPullingTime="1.146619725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:08:06.114930979 +0000 UTC m=+683.102042968" watchObservedRunningTime="2026-10-31 01:08:06.115565549 +0000 UTC m=+683.102677526"
Oct 31 01:08:19.442549 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:08:19.442514    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:08:49.449830 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:08:49.449727    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:09:06.425207 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.425176    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"]
Oct 31 01:09:06.429270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.429254    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.440124 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.440103    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-7f6d1e-openshift-default-dockercfg-fjfcz\""
Oct 31 01:09:06.442559 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.442537    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"]
Oct 31 01:09:06.547276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547237    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547277    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpc8p\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547419    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547515    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547551    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547610 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547578    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547668 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547651    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547689    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.547740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.547720    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.623814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.623771    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"]
Oct 31 01:09:06.627588 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.627567    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.640200 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.640152    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-0a6cca-openshift-default-dockercfg-jn6mf\""
Oct 31 01:09:06.640777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.640701    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"]
Oct 31 01:09:06.648786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.648758    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.648892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.648838    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.648892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.648864    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649003 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.648891    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649003 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.648918    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649303    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649371    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649392    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649408    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649507    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649562    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpc8p\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649694    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.649957 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.649915    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.650032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.650009    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.651529 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.651505    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.651909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.651889    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.671563 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.665928    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.671563 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.667878    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpc8p\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.744161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.744075    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:06.750669 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.750647    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.750749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.750684    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.750749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.750713    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.750877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.750856    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.750936 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.750893    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.751063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.751048    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.751122 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.751074    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.751170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.751154    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx6zv\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.751217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.751197    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.823642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.823595    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"]
Oct 31 01:09:06.827519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.827500    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.842771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.842715    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"]
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852496    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852535    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852644    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx6zv\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852689    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852756    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852810    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852841    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852902    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852926    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.853163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.852945    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.854702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.854569    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.854867 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.854733    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.855825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.855334    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.855825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.855594    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.856074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.855841    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.858356 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.858334    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.861063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.861046    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-8499c8-openshift-default-dockercfg-h4hkj\""
Oct 31 01:09:06.867204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.867184    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.867543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.867524    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx6zv\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.903433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.903406    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"]
Oct 31 01:09:06.908296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.908268    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.919284 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.919260    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"]
Oct 31 01:09:06.942795 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.942136    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:06.954182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954119    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.954182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954157    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.954386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954236    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.954386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954273    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.954386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954299    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.954386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954331    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnbls\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.954720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954478    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.954720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954512    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.954720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954643    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.954720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954688    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954735    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954799    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954850    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954882    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954920    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954951    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.954983    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6lc7\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:06.955038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.955004    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:06.957630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:06.957576    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-28ff08-openshift-default-dockercfg-gpfzz\""
Oct 31 01:09:07.033767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.033711    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"]
Oct 31 01:09:07.042569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.042539    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.042703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.042633    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.042703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.042670    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.055898 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.055873    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.055913    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.055940    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.055965    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056012    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056045    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056074    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056108    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056147    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056172    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056203    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6lc7\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056236    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056272    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056307    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056343    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056383    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056381    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056422    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056493    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnbls\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056505    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.056811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.056710    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.057313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057062    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.057313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057295    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.057435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057309    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.057435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057426    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.057584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057525    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.057923 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057683    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.057923 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.057783    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.058752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.058732    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.059573 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.059547    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.059958 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.059935    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.060938 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.060919    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.066098 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.066072    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.066098 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.066088    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnbls\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.066518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.066496    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.066585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.066532    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6lc7\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.144517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.144474    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:07.233070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.233032    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"]
Oct 31 01:09:07.234582 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.234553    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:07.264576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.264033    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.264576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.264095    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.264576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.264126    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.405796 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.405760    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" event={"ID":"e08a8410-136a-4178-80ce-dd599ca42048","Type":"ContainerStarted","Data":"77d37e1c01078071eb108b000cbcd0dfac1eaa448c894cef2a5357d38e8adcb2"}
Oct 31 01:09:07.405988 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.405823    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" event={"ID":"e08a8410-136a-4178-80ce-dd599ca42048","Type":"ContainerStarted","Data":"a73e832e5fdcdd67c8b138cbc214dac42e4627f145973cd725a4a0ffb7041444"}
Oct 31 01:09:07.408149 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.408120    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" event={"ID":"5303487b-1577-4432-b50d-a06e79249525","Type":"ContainerStarted","Data":"093334fc610b9cd5eb75f8f76bf74024928a987870121206cb5ffe8d424e03bb"}
Oct 31 01:09:07.408362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.408342    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" event={"ID":"5303487b-1577-4432-b50d-a06e79249525","Type":"ContainerStarted","Data":"35b95912c66954bf5c5643dae559a460be240f6143461df2cd2263a76ddf1f14"}
Oct 31 01:09:07.428430 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.428374    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podStartSLOduration=1.428358363 podStartE2EDuration="1.428358363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:07.424696439 +0000 UTC m=+744.411808419" watchObservedRunningTime="2026-10-31 01:09:07.428358363 +0000 UTC m=+744.415470339"
Oct 31 01:09:07.449966 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.449909    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podStartSLOduration=1.44989044 podStartE2EDuration="1.44989044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:07.446957838 +0000 UTC m=+744.434069815" watchObservedRunningTime="2026-10-31 01:09:07.44989044 +0000 UTC m=+744.437002417"
Oct 31 01:09:07.452331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.452308    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"]
Oct 31 01:09:07.454246 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:07.454219    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfa0bd8_3aa9_4fc7_91a1_a1c0b4c8ff92.slice/crio-c90605c480c5505a72cea84d8d37c2f9fef12e160311c3a2e058af52b5c67368 WatchSource:0}: Error finding container c90605c480c5505a72cea84d8d37c2f9fef12e160311c3a2e058af52b5c67368: Status 404 returned error can't find the container with id c90605c480c5505a72cea84d8d37c2f9fef12e160311c3a2e058af52b5c67368
Oct 31 01:09:07.456760 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.456723    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.456828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.456806    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.456871 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.456847    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.545295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.545266    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"]
Oct 31 01:09:07.548205 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:07.548175    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83bf61af_84f3_4b99_865d_e1cbd5a1918e.slice/crio-f0c4d4a5cb385e161fc2552305e4f25a1b63e8661eff341c585d5caaafbf8727 WatchSource:0}: Error finding container f0c4d4a5cb385e161fc2552305e4f25a1b63e8661eff341c585d5caaafbf8727: Status 404 returned error can't find the container with id f0c4d4a5cb385e161fc2552305e4f25a1b63e8661eff341c585d5caaafbf8727
Oct 31 01:09:07.551000 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.550958    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.551112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.551049    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.551112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.551089    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:07.744214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.744183    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:07.943065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:07.943035    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:08.421122 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.421084    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" event={"ID":"83bf61af-84f3-4b99-865d-e1cbd5a1918e","Type":"ContainerStarted","Data":"64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"}
Oct 31 01:09:08.421431 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.421407    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" event={"ID":"83bf61af-84f3-4b99-865d-e1cbd5a1918e","Type":"ContainerStarted","Data":"f0c4d4a5cb385e161fc2552305e4f25a1b63e8661eff341c585d5caaafbf8727"}
Oct 31 01:09:08.424394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.424159    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" event={"ID":"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92","Type":"ContainerStarted","Data":"ca93755bd79f7014cd5cf41390388570a779ef885e311f92b88d8e5a6127540e"}
Oct 31 01:09:08.424394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.424222    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" event={"ID":"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92","Type":"ContainerStarted","Data":"c90605c480c5505a72cea84d8d37c2f9fef12e160311c3a2e058af52b5c67368"}
Oct 31 01:09:08.447353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.447295    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podStartSLOduration=2.447282215 podStartE2EDuration="2.447282215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:08.446367996 +0000 UTC m=+745.433479974" watchObservedRunningTime="2026-10-31 01:09:08.447282215 +0000 UTC m=+745.434394191"
Oct 31 01:09:08.467008 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.466952    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podStartSLOduration=2.466936452 podStartE2EDuration="2.466936452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:08.466787776 +0000 UTC m=+745.453899754" watchObservedRunningTime="2026-10-31 01:09:08.466936452 +0000 UTC m=+745.454048429"
Oct 31 01:09:08.750394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.750321    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:08.947946 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:08.947911    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:09.145749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.145717    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:09.150869 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.150845    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:09.235726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.235703    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:09.240841 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.240817    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:09.430158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430080    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:09.430158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430129    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:09.430158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430151    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:09.430637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430173    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:09.430786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430764    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:09.430864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430820    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:09.430864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430851    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:09.430941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:09.430879    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:11.954924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:11.954887    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"]
Oct 31 01:09:11.958798 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:11.958778    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:11.966926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:11.966907    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-0a6cca\""
Oct 31 01:09:11.970555 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:11.970525    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"]
Oct 31 01:09:12.006583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.006550    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-0a6cca-dockercfg-wxkhl\""
Oct 31 01:09:12.014229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.014205    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.014346 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.014251    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p65cj\" (UniqueName: \"kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj\") pod \"payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.115725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.115691    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.115725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.115728    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p65cj\" (UniqueName: \"kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj\") pod \"payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.116375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.116355    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.134521 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.134491    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p65cj\" (UniqueName: \"kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj\") pod \"payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.276182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.276104    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:12.577319 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.577228    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"]
Oct 31 01:09:12.580515 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:12.580481    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448f6a3f_07c9_4381_a356_ec466a740f46.slice/crio-8d35f355e98b149df91d01678df1d2aef0eb1a4217487905adc97e7c6515d154 WatchSource:0}: Error finding container 8d35f355e98b149df91d01678df1d2aef0eb1a4217487905adc97e7c6515d154: Status 404 returned error can't find the container with id 8d35f355e98b149df91d01678df1d2aef0eb1a4217487905adc97e7c6515d154
Oct 31 01:09:12.582811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:12.582796    8029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:09:13.122855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.122817    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"]
Oct 31 01:09:13.128651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.128629    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"]
Oct 31 01:09:13.128771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.128725    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.132212 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.132193    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.136828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.136803    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"]
Oct 31 01:09:13.137757 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.137736    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-7f6d1e-dockercfg-65c9z\""
Oct 31 01:09:13.141979 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.140506    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"]
Oct 31 01:09:13.164205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.164176    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-7f6d1e\""
Oct 31 01:09:13.232035 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.232008    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nr9m\" (UniqueName: \"kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m\") pod \"payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.232202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.232050    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27mx5\" (UniqueName: \"kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5\") pod \"payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.232202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.232083    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.232326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.232212    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.333438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.333400    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nr9m\" (UniqueName: \"kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m\") pod \"payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.333640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.333457    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27mx5\" (UniqueName: \"kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5\") pod \"payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.333640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.333486    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.333640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.333511    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.334261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.334240    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.334511 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.334493    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.341928 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.341906    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nr9m\" (UniqueName: \"kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m\") pod \"payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.347299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.347275    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27mx5\" (UniqueName: \"kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5\") pod \"payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.451395 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.451326    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" event={"ID":"448f6a3f-07c9-4381-a356-ec466a740f46","Type":"ContainerStarted","Data":"8d35f355e98b149df91d01678df1d2aef0eb1a4217487905adc97e7c6515d154"}
Oct 31 01:09:13.457522 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.457498    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:13.460389 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.460367    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:13.680843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.680811    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"]
Oct 31 01:09:13.686131 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.686102    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:13.695268 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.695250    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-acdaed\""
Oct 31 01:09:13.702911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.701889    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"]
Oct 31 01:09:13.740076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.740041    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:13.740299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.740242    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qk7n\" (UniqueName: \"kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n\") pod \"payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:13.755014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.754988    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"]
Oct 31 01:09:13.757354 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.757330    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"]
Oct 31 01:09:13.758925 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:13.758896    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8b9992_fca7_49f1_b464_b445d8054d5d.slice/crio-796fdcac8b34aa55fefbbf8622a3e23329a29f1300ae9fc8f7be10fe888ad7f6 WatchSource:0}: Error finding container 796fdcac8b34aa55fefbbf8622a3e23329a29f1300ae9fc8f7be10fe888ad7f6: Status 404 returned error can't find the container with id 796fdcac8b34aa55fefbbf8622a3e23329a29f1300ae9fc8f7be10fe888ad7f6
Oct 31 01:09:13.760679 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:13.760659    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafc302a_4d33_44df_b0b2_29fdb44a84b8.slice/crio-32325d478f867c2464df82cc8ff3d7cbda8282ed1d330068563beee6b8899c69 WatchSource:0}: Error finding container 32325d478f867c2464df82cc8ff3d7cbda8282ed1d330068563beee6b8899c69: Status 404 returned error can't find the container with id 32325d478f867c2464df82cc8ff3d7cbda8282ed1d330068563beee6b8899c69
Oct 31 01:09:13.841645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.841585    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:13.841796 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.841741    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qk7n\" (UniqueName: \"kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n\") pod \"payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:13.842218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.842200    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:13.851650 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:13.851626    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qk7n\" (UniqueName: \"kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n\") pod \"payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:14.000722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.000637    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:14.291557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.291483    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"]
Oct 31 01:09:14.294473 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:14.294406    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef9a1a2_af7b_4e49_b6b2_6bd0588b4dc9.slice/crio-5d0b208093653346068508cdc179d25b6d55bb2ecb81527ac05584a1fa9f4398 WatchSource:0}: Error finding container 5d0b208093653346068508cdc179d25b6d55bb2ecb81527ac05584a1fa9f4398: Status 404 returned error can't find the container with id 5d0b208093653346068508cdc179d25b6d55bb2ecb81527ac05584a1fa9f4398
Oct 31 01:09:14.379565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.379540    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"]
Oct 31 01:09:14.384884 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.383731    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.391256 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.391236    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-487a6c-dockercfg-82tzt\""
Oct 31 01:09:14.391377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.391298    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-487a6c\""
Oct 31 01:09:14.393030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.393007    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"]
Oct 31 01:09:14.451277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.451241    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2hpn\" (UniqueName: \"kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn\") pod \"payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.453811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.451347    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.464854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.464822    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" event={"ID":"5b8b9992-fca7-49f1-b464-b445d8054d5d","Type":"ContainerStarted","Data":"796fdcac8b34aa55fefbbf8622a3e23329a29f1300ae9fc8f7be10fe888ad7f6"}
Oct 31 01:09:14.470670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.470643    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" event={"ID":"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9","Type":"ContainerStarted","Data":"5d0b208093653346068508cdc179d25b6d55bb2ecb81527ac05584a1fa9f4398"}
Oct 31 01:09:14.474685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.474660    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" event={"ID":"bafc302a-4d33-44df-b0b2-29fdb44a84b8","Type":"ContainerStarted","Data":"32325d478f867c2464df82cc8ff3d7cbda8282ed1d330068563beee6b8899c69"}
Oct 31 01:09:14.552376 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.552293    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.552530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.552446    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2hpn\" (UniqueName: \"kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn\") pod \"payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.553275 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.553248    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.565505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.565479    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2hpn\" (UniqueName: \"kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn\") pod \"payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.707117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.706991    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:14.771640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.771141    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"]
Oct 31 01:09:14.796126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.786810    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"]
Oct 31 01:09:14.796126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.787059    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:14.798210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.798188    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-28ff08-dockercfg-gnbpc\""
Oct 31 01:09:14.799148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.799123    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-28ff08\""
Oct 31 01:09:14.877575 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.877538    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wqng\" (UniqueName: \"kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng\") pod \"payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:14.877779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.877693    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:14.980881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.978895    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wqng\" (UniqueName: \"kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng\") pod \"payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:14.980881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.979026    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:14.980881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:14.980025    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:15.011838 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.011379    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"]
Oct 31 01:09:15.013379 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:15.013344    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da29a69_9b53_4506_8db0_4758c504af67.slice/crio-f788f66d4a0b76581e4bd58dba5e6640388fe2b3f436fd5ce33b0ee5363c57f0 WatchSource:0}: Error finding container f788f66d4a0b76581e4bd58dba5e6640388fe2b3f436fd5ce33b0ee5363c57f0: Status 404 returned error can't find the container with id f788f66d4a0b76581e4bd58dba5e6640388fe2b3f436fd5ce33b0ee5363c57f0
Oct 31 01:09:15.016074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.016058    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wqng\" (UniqueName: \"kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng\") pod \"payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:15.114712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.114400    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:15.474568 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.472765    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"]
Oct 31 01:09:15.482401 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:15.482364    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957a9635_2910_4b1b_9cc9_87e7672971f5.slice/crio-a62f855cb3fe743b11083424c2bf73666e3fbaef967a0273fa1075bb45b6357b WatchSource:0}: Error finding container a62f855cb3fe743b11083424c2bf73666e3fbaef967a0273fa1075bb45b6357b: Status 404 returned error can't find the container with id a62f855cb3fe743b11083424c2bf73666e3fbaef967a0273fa1075bb45b6357b
Oct 31 01:09:15.503904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.503850    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" event={"ID":"6da29a69-9b53-4506-8db0-4758c504af67","Type":"ContainerStarted","Data":"f788f66d4a0b76581e4bd58dba5e6640388fe2b3f436fd5ce33b0ee5363c57f0"}
Oct 31 01:09:15.650654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.642392    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"]
Oct 31 01:09:15.663623 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.658754    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"]
Oct 31 01:09:15.663623 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.658873    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.744260 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.744171    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-8499c8-dockercfg-gcnzp\""
Oct 31 01:09:15.744437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.744310    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-8499c8\""
Oct 31 01:09:15.755660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.746268    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbgwx\" (UniqueName: \"kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx\") pod \"payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.755660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.746350    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.854672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.854545    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgwx\" (UniqueName: \"kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx\") pod \"payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.854672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.854654    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.856013 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.855976    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.864627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.864583    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbgwx\" (UniqueName: \"kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx\") pod \"payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:15.987475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:15.987427    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:16.368794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:16.368765    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"]
Oct 31 01:09:16.515644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:16.515580    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" event={"ID":"957a9635-2910-4b1b-9cc9-87e7672971f5","Type":"ContainerStarted","Data":"a62f855cb3fe743b11083424c2bf73666e3fbaef967a0273fa1075bb45b6357b"}
Oct 31 01:09:16.519282 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:16.519254    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" event={"ID":"32b93a2d-9fce-43a7-8ba0-d9e10053db09","Type":"ContainerStarted","Data":"e76fc7ccb2c78051744dfba91e8751251c43ea6f7e1b6c5ab0e4df1a68932209"}
Oct 31 01:09:18.539493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.539417    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" event={"ID":"957a9635-2910-4b1b-9cc9-87e7672971f5","Type":"ContainerStarted","Data":"b728f0bd2e62b70118895f063017471259f54bc6838c1841c583dd8c78a7e90e"}
Oct 31 01:09:18.540318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.540295    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:18.544314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.544250    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" event={"ID":"448f6a3f-07c9-4381-a356-ec466a740f46","Type":"ContainerStarted","Data":"eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb"}
Oct 31 01:09:18.544686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.544492    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:18.547341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.546819    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" event={"ID":"6da29a69-9b53-4506-8db0-4758c504af67","Type":"ContainerStarted","Data":"44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711"}
Oct 31 01:09:18.547341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.547291    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:18.558295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.556578    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" podStartSLOduration=1.856577988 podStartE2EDuration="4.556560467s" totalImagesPullingTime="2.699982479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:18.556160133 +0000 UTC m=+755.543272111" watchObservedRunningTime="2026-10-31 01:09:18.556560467 +0000 UTC m=+755.543672446"
Oct 31 01:09:18.574375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:18.574300    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" podStartSLOduration=2.310087795 podStartE2EDuration="7.574285764s" totalImagesPullingTime="5.264197969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:18.574265543 +0000 UTC m=+755.561377521" watchObservedRunningTime="2026-10-31 01:09:18.574285764 +0000 UTC m=+755.561397741"
Oct 31 01:09:19.445515 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:09:19.445474    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:09:19.554040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.554007    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" event={"ID":"5b8b9992-fca7-49f1-b464-b445d8054d5d","Type":"ContainerStarted","Data":"63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d"}
Oct 31 01:09:19.554455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.554087    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:19.555657 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.555627    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" event={"ID":"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9","Type":"ContainerStarted","Data":"6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9"}
Oct 31 01:09:19.555789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.555684    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:19.556999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.556975    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" event={"ID":"bafc302a-4d33-44df-b0b2-29fdb44a84b8","Type":"ContainerStarted","Data":"cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d"}
Oct 31 01:09:19.557143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.557055    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:19.558690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.558665    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" event={"ID":"32b93a2d-9fce-43a7-8ba0-d9e10053db09","Type":"ContainerStarted","Data":"94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894"}
Oct 31 01:09:19.558902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.558884    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:19.570453 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.570416    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" podStartSLOduration=1.5873463939999999 podStartE2EDuration="6.57040535s" totalImagesPullingTime="4.983058956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:19.569219051 +0000 UTC m=+756.556331030" watchObservedRunningTime="2026-10-31 01:09:19.57040535 +0000 UTC m=+756.557517326"
Oct 31 01:09:19.571471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.571438    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" podStartSLOduration=2.678113142 podStartE2EDuration="5.571428667s" totalImagesPullingTime="2.893315525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:18.594687691 +0000 UTC m=+755.581799668" watchObservedRunningTime="2026-10-31 01:09:19.571428667 +0000 UTC m=+756.558540643"
Oct 31 01:09:19.586659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.586625    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" podStartSLOduration=1.9367391999999999 podStartE2EDuration="6.58661434s" totalImagesPullingTime="4.64987514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:19.584716497 +0000 UTC m=+756.571828474" watchObservedRunningTime="2026-10-31 01:09:19.58661434 +0000 UTC m=+756.573726307"
Oct 31 01:09:19.603367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.603330    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" podStartSLOduration=1.6613704999999999 podStartE2EDuration="6.60332065s" totalImagesPullingTime="4.94195015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:19.601096811 +0000 UTC m=+756.588208788" watchObservedRunningTime="2026-10-31 01:09:19.60332065 +0000 UTC m=+756.590432646"
Oct 31 01:09:19.617516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:19.617483    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" podStartSLOduration=2.3023565440000002 podStartE2EDuration="4.617473451s" totalImagesPullingTime="2.315116907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:19.61628471 +0000 UTC m=+756.603396687" watchObservedRunningTime="2026-10-31 01:09:19.617473451 +0000 UTC m=+756.604585420"
Oct 31 01:09:29.563517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:29.562669    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:09:30.569809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:30.569775    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:09:30.570328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:30.569959    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:09:30.570328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:30.570268    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:09:30.570451 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:30.570348    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:09:30.570957 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:30.570935    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:09:30.571041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:30.571003    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:09:32.773813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.773781    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"]
Oct 31 01:09:32.800230 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.800156    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:32.814468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.814378    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-487a6c\"/\"default-dockercfg-9hlbr\""
Oct 31 01:09:32.814693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.814550    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-487a6c\"/\"simulator-w3-487a6c-kserve-self-signed-certs\""
Oct 31 01:09:32.816638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.814875    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-487a6c\"/\"openshift-service-ca.crt\""
Oct 31 01:09:32.816638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.814960    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-487a6c\"/\"kube-root-ca.crt\""
Oct 31 01:09:32.818057 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.818002    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"]
Oct 31 01:09:32.902345 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.902261    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:32.902345 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.902313    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:32.902630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.902464    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zmcv\" (UniqueName: \"kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:32.903846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.902518    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:32.903846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.903747    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:32.903846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:32.903821    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.006631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.006289    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.006631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.006352    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.006631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.006443    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zmcv\" (UniqueName: \"kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.006631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.006490    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.006631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.006575    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.007075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.006664    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.007631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.007175    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.007631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.007453    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.007778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.007740    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.011698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.010739    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.011815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.011765    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.015884 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.015858    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zmcv\" (UniqueName: \"kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv\") pod \"simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") " pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.145171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.145119    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:33.828839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.468180    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"]
Oct 31 01:09:33.828839 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:33.469545    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3288e4f3_0914_457e_bb66_ba36c4c2edc8.slice/crio-0440bbc07c7679f3c271471267ff25c8df7e38da08535c666955a1bd9b3c88e3 WatchSource:0}: Error finding container 0440bbc07c7679f3c271471267ff25c8df7e38da08535c666955a1bd9b3c88e3: Status 404 returned error can't find the container with id 0440bbc07c7679f3c271471267ff25c8df7e38da08535c666955a1bd9b3c88e3
Oct 31 01:09:33.850490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.850454    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" event={"ID":"3288e4f3-0914-457e-bb66-ba36c4c2edc8","Type":"ContainerStarted","Data":"93a008dfe697355201a758ba5f7c886c3e6af1f50922473c02080672a73937a6"}
Oct 31 01:09:33.850662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.850510    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" event={"ID":"3288e4f3-0914-457e-bb66-ba36c4c2edc8","Type":"ContainerStarted","Data":"0440bbc07c7679f3c271471267ff25c8df7e38da08535c666955a1bd9b3c88e3"}
Oct 31 01:09:33.871294 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:33.871252    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podStartSLOduration=1.871238132 podStartE2EDuration="1.871238132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:33.869801603 +0000 UTC m=+770.856913578" watchObservedRunningTime="2026-10-31 01:09:33.871238132 +0000 UTC m=+770.858350107"
Oct 31 01:09:34.145671 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.145596    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:34.858292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.858262    8029 generic.go:356] "Generic (PLEG): container finished" podID="5303487b-1577-4432-b50d-a06e79249525" containerID="093334fc610b9cd5eb75f8f76bf74024928a987870121206cb5ffe8d424e03bb" exitCode=0
Oct 31 01:09:34.858869 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.858343    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" event={"ID":"5303487b-1577-4432-b50d-a06e79249525","Type":"ContainerDied","Data":"093334fc610b9cd5eb75f8f76bf74024928a987870121206cb5ffe8d424e03bb"}
Oct 31 01:09:34.858869 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.858826    8029 scope.go:122] "RemoveContainer" containerID="093334fc610b9cd5eb75f8f76bf74024928a987870121206cb5ffe8d424e03bb"
Oct 31 01:09:34.860301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.860274    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:34.860393 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.860342    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:34.860393 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.860377    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:34.860477 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.860456    8029 generic.go:356] "Generic (PLEG): container finished" podID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerID="64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378" exitCode=0
Oct 31 01:09:34.860948 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.860929    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" event={"ID":"83bf61af-84f3-4b99-865d-e1cbd5a1918e","Type":"ContainerDied","Data":"64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"}
Oct 31 01:09:34.861254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.861234    8029 scope.go:122] "RemoveContainer" containerID="64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"
Oct 31 01:09:34.862569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.862541    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:34.862681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.862651    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:34.862732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:34.862706    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:35.868763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.868733    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" event={"ID":"83bf61af-84f3-4b99-865d-e1cbd5a1918e","Type":"ContainerStarted","Data":"1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1"}
Oct 31 01:09:35.870266 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.870245    8029 generic.go:356] "Generic (PLEG): container finished" podID="e08a8410-136a-4178-80ce-dd599ca42048" containerID="77d37e1c01078071eb108b000cbcd0dfac1eaa448c894cef2a5357d38e8adcb2" exitCode=0
Oct 31 01:09:35.870460 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.870324    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" event={"ID":"e08a8410-136a-4178-80ce-dd599ca42048","Type":"ContainerDied","Data":"77d37e1c01078071eb108b000cbcd0dfac1eaa448c894cef2a5357d38e8adcb2"}
Oct 31 01:09:35.870858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.870838    8029 scope.go:122] "RemoveContainer" containerID="77d37e1c01078071eb108b000cbcd0dfac1eaa448c894cef2a5357d38e8adcb2"
Oct 31 01:09:35.872312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.872278    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:35.872693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.872668    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:35.872779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.872733    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:35.873436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:35.873415    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" event={"ID":"5303487b-1577-4432-b50d-a06e79249525","Type":"ContainerStarted","Data":"f7705e9036e1e00bb902d4234c576470d6bdc73483ea0127860fb772664c48ba"}
Oct 31 01:09:36.243428 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.241391    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:36.247349 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.247315    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 31 01:09:36.247516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.247389    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 31 01:09:36.747981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.747949    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:36.750755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.750724    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 31 01:09:36.751009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.750981    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 31 01:09:36.942638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.941251    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" event={"ID":"e08a8410-136a-4178-80ce-dd599ca42048","Type":"ContainerStarted","Data":"6d602f1615bfaf5502bd86c375f265d4c4f1eb5a817bd74bccf52042a1730cfd"}
Oct 31 01:09:36.957064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:36.954929    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:37.282802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:37.269331    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 31 01:09:37.282802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:37.269402    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 31 01:09:37.750289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:37.749867    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 31 01:09:37.750289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:37.749926    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 31 01:09:37.951234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:37.951201    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 31 01:09:37.951870 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:37.951823    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 31 01:09:38.274625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:38.270808    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 31 01:09:38.274625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:38.270883    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 31 01:09:38.746625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:38.746019    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 31 01:09:38.746625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:38.746077    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 31 01:09:38.944633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:38.944028    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 31 01:09:38.944633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:38.944099    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 31 01:09:39.236462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.236017    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 31 01:09:39.236462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.236079    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 31 01:09:39.441698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.434718    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:39.447511 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.446791    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:39.451626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.447838    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:39.745627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.745050    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 31 01:09:39.745627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.745123    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 31 01:09:39.948482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.948028    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 31 01:09:39.948482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:39.948089    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 31 01:09:40.239079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.239040    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 31 01:09:40.239689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.239654    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 31 01:09:40.441922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.441887    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"]
Oct 31 01:09:40.573947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.493705    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:09:40.573947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.493772    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:09:40.573947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.493890    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.642638    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzr8m\" (UniqueName: \"kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.642688    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.642848    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.642943    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.643004    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.643029    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.643187    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7f6d1e\"/\"kube-root-ca.crt\""
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.643248    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7f6d1e\"/\"openshift-service-ca.crt\""
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.643313    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7f6d1e\"/\"simulator-w1-7f6d1e-kserve-self-signed-certs\""
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.643214    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7f6d1e\"/\"default-dockercfg-mqk8h\""
Oct 31 01:09:40.668666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.646731    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"]
Oct 31 01:09:40.748627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.747862    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.748627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.747979    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.748627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.748045    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.748627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.748082    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.748627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.748119    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzr8m\" (UniqueName: \"kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.748627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.748216    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.750111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.750076    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.750397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.750372    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.753640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.752497    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.753640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.752879    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.753640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.753543    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 31 01:09:40.753640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.753619    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 31 01:09:40.762424 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.762385    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzr8m\" (UniqueName: \"kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.772674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.772646    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs\") pod \"simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") " pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:40.948519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.948061    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 31 01:09:40.948519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:40.948124    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 31 01:09:41.073151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.073107    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:41.236736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.236263    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 31 01:09:41.236736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.236321    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 31 01:09:41.438716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.438666    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"]
Oct 31 01:09:41.749266 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.749185    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:41.750117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.750094    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:09:41.943269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.943236    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 31 01:09:41.943422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:41.943293    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 31 01:09:42.031763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.022856    8029 generic.go:356] "Generic (PLEG): container finished" podID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerID="ca93755bd79f7014cd5cf41390388570a779ef885e311f92b88d8e5a6127540e" exitCode=0
Oct 31 01:09:42.031763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.022929    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" event={"ID":"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92","Type":"ContainerDied","Data":"ca93755bd79f7014cd5cf41390388570a779ef885e311f92b88d8e5a6127540e"}
Oct 31 01:09:42.031763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.023438    8029 scope.go:122] "RemoveContainer" containerID="ca93755bd79f7014cd5cf41390388570a779ef885e311f92b88d8e5a6127540e"
Oct 31 01:09:42.034928 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.034894    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:42.035021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.034974    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:42.035021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.035009    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:09:42.036338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.036313    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" event={"ID":"23154512-832c-4b95-bf6e-872707c8ec20","Type":"ContainerStarted","Data":"5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb"}
Oct 31 01:09:42.036439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.036351    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" event={"ID":"23154512-832c-4b95-bf6e-872707c8ec20","Type":"ContainerStarted","Data":"b195a9abbd89a3e429c8fa684f6fe7b085ad8c0a9258fbc7f573f46e1d4cb41b"}
Oct 31 01:09:42.075996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.075011    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:42.100627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.090360    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podStartSLOduration=2.090337147 podStartE2EDuration="2.090337147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:42.088840333 +0000 UTC m=+779.075952310" watchObservedRunningTime="2026-10-31 01:09:42.090337147 +0000 UTC m=+779.077449131"
Oct 31 01:09:42.263809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.262299    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:42.265661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.264926    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:09:42.950503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.950473    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:42.951748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:42.951724    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:09:43.045164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:43.045130    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" event={"ID":"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92","Type":"ContainerStarted","Data":"9d11bb4f50dc3ebda1173575dbd694fdcd30d410288a6e0c2daa29634b1a624a"}
Oct 31 01:09:43.144758 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:43.144726    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:43.144921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:43.144907    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 31 01:09:43.144973 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:43.144952    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 31 01:09:43.145207 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:43.145195    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:43.158569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:43.158544    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:44.061268 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:44.061240    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:09:44.145325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:44.145288    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 31 01:09:44.145483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:44.145366    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 31 01:09:45.145325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:45.145295    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 31 01:09:45.145737 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:45.145358    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 31 01:09:46.145359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:46.145325    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 31 01:09:46.145774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:46.145387    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 31 01:09:47.145174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:47.145143    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 31 01:09:47.145342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:47.145202    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 31 01:09:48.145223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.145182    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 31 01:09:48.145650 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.145259    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 31 01:09:48.490262 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.490185    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"]
Oct 31 01:09:48.495181 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.495160    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.500166 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.500141    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8499c8\"/\"default-dockercfg-qcw2m\""
Oct 31 01:09:48.502744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.502723    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8499c8\"/\"simulator-w4-8499c8-kserve-self-signed-certs\""
Oct 31 01:09:48.504590 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.504567    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-8499c8\"/\"kube-root-ca.crt\""
Oct 31 01:09:48.504700 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.504644    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-8499c8\"/\"openshift-service-ca.crt\""
Oct 31 01:09:48.507287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.507263    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"]
Oct 31 01:09:48.574128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.574099    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.574288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.574138    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.574288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.574244    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkst2\" (UniqueName: \"kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.574388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.574373    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.574427 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.574399    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.574462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.574441    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.675568 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.675526    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.675769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.675662    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkst2\" (UniqueName: \"kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.675836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.675798    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.675896 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.675833    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.675955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.675912    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.676018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.675968    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.676245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.676221    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.676383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.676338    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.676631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.676562    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.679126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.679100    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.684830 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.684810    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkst2\" (UniqueName: \"kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.685795 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.685769    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm\") pod \"simulator-w4-8499c8-kserve-6cb6c79696-s9ddp\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") " pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:48.808462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:48.808369    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:49.097507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:49.097483    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"]
Oct 31 01:09:49.099711 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:09:49.099674    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7132e0fc_e020_4ce9_9ca3_732da214abb6.slice/crio-e54dbb333a1f4c4bedb532b33928127d8441c40dcf28022827b43c9e56edcd54 WatchSource:0}: Error finding container e54dbb333a1f4c4bedb532b33928127d8441c40dcf28022827b43c9e56edcd54: Status 404 returned error can't find the container with id e54dbb333a1f4c4bedb532b33928127d8441c40dcf28022827b43c9e56edcd54
Oct 31 01:09:49.150079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:49.149994    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:49.150672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:49.150649    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:49.153286 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:49.153265    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:09:49.476697 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:09:49.476652    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:09:50.084803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:50.084768    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" event={"ID":"7132e0fc-e020-4ce9-9ca3-732da214abb6","Type":"ContainerStarted","Data":"57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a"}
Oct 31 01:09:50.084803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:50.084806    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" event={"ID":"7132e0fc-e020-4ce9-9ca3-732da214abb6","Type":"ContainerStarted","Data":"e54dbb333a1f4c4bedb532b33928127d8441c40dcf28022827b43c9e56edcd54"}
Oct 31 01:09:50.104009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:50.103945    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podStartSLOduration=2.103926477 podStartE2EDuration="2.103926477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:09:50.101331102 +0000 UTC m=+787.088443080" watchObservedRunningTime="2026-10-31 01:09:50.103926477 +0000 UTC m=+787.091038454"
Oct 31 01:09:50.808996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:50.808945    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:51.073976 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:51.073901    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:51.086858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:51.086834    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:51.102583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:51.102557    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:09:58.809261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:58.809219    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:58.821717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:58.821692    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:09:59.145185 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:09:59.145154    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:10:03.293470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.293434    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"]
Oct 31 01:10:03.297568 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.297552    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.314024 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.314004    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-acdaed\"/\"kube-root-ca.crt\""
Oct 31 01:10:03.314287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.314215    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-acdaed\"/\"openshift-service-ca.crt\""
Oct 31 01:10:03.315326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.315307    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-acdaed\"/\"default-dockercfg-dhhtn\""
Oct 31 01:10:03.315437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.315333    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-acdaed\"/\"premium-w0-acdaed-kserve-self-signed-certs\""
Oct 31 01:10:03.317502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.317482    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"]
Oct 31 01:10:03.350768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.350734    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.350914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.350795    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.350914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.350901    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.350985 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.350926    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.350985 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.350964    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8smss\" (UniqueName: \"kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.351130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.351117    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.451844 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.451815    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.451878    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.451934    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.451958    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.451995    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8smss\" (UniqueName: \"kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.452113    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.452283    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.452351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.452332    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.453361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.452537    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.455858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.455830    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.456398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.456356    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.461938 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.461911    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8smss\" (UniqueName: \"kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss\") pod \"premium-w0-acdaed-kserve-869b8dbddd-ds6g9\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") " pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.610433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.610402    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:03.897442 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:03.897419    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"]
Oct 31 01:10:03.899975 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:10:03.899951    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25a79474_3747_4ba2_b31d_754962b8f9e1.slice/crio-9cc7ae6bd67c7ed6759f01987f5690ad2abf7af18befdf8d7b6479abb71616e2 WatchSource:0}: Error finding container 9cc7ae6bd67c7ed6759f01987f5690ad2abf7af18befdf8d7b6479abb71616e2: Status 404 returned error can't find the container with id 9cc7ae6bd67c7ed6759f01987f5690ad2abf7af18befdf8d7b6479abb71616e2
Oct 31 01:10:04.163070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:04.162983    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" event={"ID":"25a79474-3747-4ba2-b31d-754962b8f9e1","Type":"ContainerStarted","Data":"42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07"}
Oct 31 01:10:04.163070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:04.163025    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" event={"ID":"25a79474-3747-4ba2-b31d-754962b8f9e1","Type":"ContainerStarted","Data":"9cc7ae6bd67c7ed6759f01987f5690ad2abf7af18befdf8d7b6479abb71616e2"}
Oct 31 01:10:04.188553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:04.188505    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podStartSLOduration=1.188491626 podStartE2EDuration="1.188491626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:04.185419908 +0000 UTC m=+801.172531881" watchObservedRunningTime="2026-10-31 01:10:04.188491626 +0000 UTC m=+801.175603602"
Oct 31 01:10:04.611514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:04.611481    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:06.873327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.873291    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"]
Oct 31 01:10:06.877334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.877318    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.893187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.893157    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"]
Oct 31 01:10:06.904059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.904029    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-acdaed-openshift-default-dockercfg-swmgr\""
Oct 31 01:10:06.991434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991393    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991435    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991453    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991520    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991573    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991631    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991670    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991718    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:06.991997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:06.991832    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525nw\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092445    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092501    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092524    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092561    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092644    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525nw\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092729    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092774    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092798    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.092956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092826    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.093164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092984    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.093164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.092997    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.093280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.093180    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.093280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.093236    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.093751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.093730    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.095177 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.095156    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.095294 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.095275    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.103178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.103146    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.105971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.105948    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-525nw\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.192630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.192522    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:07.473684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.473657    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"]
Oct 31 01:10:07.475216 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:10:07.475182    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bb1ced_152a_488d_8b65_d70892db4c2e.slice/crio-b66b1ec610e1e1a27470c92198367ad7472ce2d79b1940408e8b331f8a151697 WatchSource:0}: Error finding container b66b1ec610e1e1a27470c92198367ad7472ce2d79b1940408e8b331f8a151697: Status 404 returned error can't find the container with id b66b1ec610e1e1a27470c92198367ad7472ce2d79b1940408e8b331f8a151697
Oct 31 01:10:07.477837 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.477803    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:07.477925 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.477889    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:07.477965 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:07.477928    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:08.186781 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:08.186740    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" event={"ID":"05bb1ced-152a-488d-8b65-d70892db4c2e","Type":"ContainerStarted","Data":"c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f"}
Oct 31 01:10:08.187191 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:08.186804    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" event={"ID":"05bb1ced-152a-488d-8b65-d70892db4c2e","Type":"ContainerStarted","Data":"b66b1ec610e1e1a27470c92198367ad7472ce2d79b1940408e8b331f8a151697"}
Oct 31 01:10:08.193140 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:08.193119    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:08.194462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:08.194438    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 01:10:08.194555 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:08.194488    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 01:10:08.209629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:08.209571    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podStartSLOduration=2.209559144 podStartE2EDuration="2.209559144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:08.20740101 +0000 UTC m=+805.194512986" watchObservedRunningTime="2026-10-31 01:10:08.209559144 +0000 UTC m=+805.196671120"
Oct 31 01:10:09.201018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:09.200977    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 01:10:09.201741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:09.201707    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 01:10:10.195627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:10.195563    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 01:10:10.195829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:10.195643    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 01:10:11.203627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:11.201884    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 01:10:11.203627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:11.201948    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 01:10:12.202040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:12.202002    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 01:10:12.202211 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:12.202065    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 01:10:13.201775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:13.201714    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 01:10:13.202267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:13.201789    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 01:10:13.610927 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:13.610894    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:13.624425 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:13.624401    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:14.272162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:14.272133    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:10:14.986472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:14.986437    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:14.986920 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:14.986890    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:14.987827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:14.987808    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:10:15.872993 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.872967    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"]
Oct 31 01:10:15.876978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.876962    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:15.881747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.881727    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-28ff08\"/\"default-dockercfg-6kts9\""
Oct 31 01:10:15.885292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.885273    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-28ff08\"/\"premium-w5-28ff08-kserve-self-signed-certs\""
Oct 31 01:10:15.889586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.889564    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"]
Oct 31 01:10:15.889915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.889890    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-28ff08\"/\"kube-root-ca.crt\""
Oct 31 01:10:15.889998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:15.889960    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-28ff08\"/\"openshift-service-ca.crt\""
Oct 31 01:10:16.030353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.030312    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.030532 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.030506    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.030577 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.030562    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.030724 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.030705    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.030784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.030754    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5277w\" (UniqueName: \"kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.030858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.030835    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132247    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132292    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5277w\" (UniqueName: \"kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132310    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132632 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132347    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132632 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132452    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132632 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132497    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132777    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132865 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132849    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.132865 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.132860    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.134734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.134717    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.135319 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.135299    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.140490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.140471    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5277w\" (UniqueName: \"kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w\") pod \"premium-w5-28ff08-kserve-648ffb4f99-grpkr\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") " pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.213725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.213690    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:16.495170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:16.495145    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"]
Oct 31 01:10:16.497470 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:10:16.497444    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c4c3bf9_e242_4933_b54e_c38c035fced3.slice/crio-13be766541065162ed55d85fdb61eb835e503af3b8b4616dcb89283ba95c5a16 WatchSource:0}: Error finding container 13be766541065162ed55d85fdb61eb835e503af3b8b4616dcb89283ba95c5a16: Status 404 returned error can't find the container with id 13be766541065162ed55d85fdb61eb835e503af3b8b4616dcb89283ba95c5a16
Oct 31 01:10:17.295405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:17.295357    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" event={"ID":"2c4c3bf9-e242-4933-b54e-c38c035fced3","Type":"ContainerStarted","Data":"917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e"}
Oct 31 01:10:17.295924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:17.295414    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" event={"ID":"2c4c3bf9-e242-4933-b54e-c38c035fced3","Type":"ContainerStarted","Data":"13be766541065162ed55d85fdb61eb835e503af3b8b4616dcb89283ba95c5a16"}
Oct 31 01:10:17.331857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:17.331803    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podStartSLOduration=2.331789778 podStartE2EDuration="2.331789778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:17.329867661 +0000 UTC m=+814.316979639" watchObservedRunningTime="2026-10-31 01:10:17.331789778 +0000 UTC m=+814.318901753"
Oct 31 01:10:18.213998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:18.213965    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:19.442763 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:10:19.442730    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:10:21.485373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.485339    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"]
Oct 31 01:10:21.490468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.490446    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.494766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.494748    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-1b8d05-openshift-default-dockercfg-q8z4w\""
Oct 31 01:10:21.497338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497314    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497483    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497528    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497595    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497647    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497720    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vzpq\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497763    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497853    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.498315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.497934    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.513018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.512982    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"]
Oct 31 01:10:21.599011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.598978    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599034    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599052    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599075    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599121    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599143    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599217    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzpq\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599260    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599294    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599434    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599471    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599748    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599774    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.599882 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.599865    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.601494 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.601470    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.601846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.601823    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.607545 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.607524    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.607920 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.607901    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vzpq\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:21.808661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:21.808559    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:22.094291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.094206    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"]
Oct 31 01:10:22.097152 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:10:22.097128    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf65f8b_7463_4c96_87ab_80e5d279210d.slice/crio-f3764d0b0de010120cec4b96c975a9ecb988a2c8ee01c19fa24c15ae2decec5d WatchSource:0}: Error finding container f3764d0b0de010120cec4b96c975a9ecb988a2c8ee01c19fa24c15ae2decec5d: Status 404 returned error can't find the container with id f3764d0b0de010120cec4b96c975a9ecb988a2c8ee01c19fa24c15ae2decec5d
Oct 31 01:10:22.099460 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.099426    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:22.099548 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.099494    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:22.099637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.099543    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:22.325992 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.325953    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" event={"ID":"7cf65f8b-7463-4c96-87ab-80e5d279210d","Type":"ContainerStarted","Data":"a6b5accd13235ac4c2ec1f02c72ae82e678fbfa388120ad11c88cc01cc64fb6d"}
Oct 31 01:10:22.326162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.326004    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" event={"ID":"7cf65f8b-7463-4c96-87ab-80e5d279210d","Type":"ContainerStarted","Data":"f3764d0b0de010120cec4b96c975a9ecb988a2c8ee01c19fa24c15ae2decec5d"}
Oct 31 01:10:22.349444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.349333    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podStartSLOduration=1.349319918 podStartE2EDuration="1.349319918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:22.349109091 +0000 UTC m=+819.336221064" watchObservedRunningTime="2026-10-31 01:10:22.349319918 +0000 UTC m=+819.336431894"
Oct 31 01:10:22.808923 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:22.808879    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:23.811167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:23.811134    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 01:10:23.811541 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:23.811187    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 01:10:24.809654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:24.809580    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 01:10:24.809828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:24.809672    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 01:10:25.809947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:25.809905    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 01:10:25.810348 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:25.809958    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 01:10:26.214014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:26.213977    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:26.227069 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:26.227043    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:26.360861 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:26.360833    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:10:26.809121 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:26.809089    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 01:10:26.809298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:26.809155    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 01:10:27.809020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:27.808979    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 31 01:10:27.809664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:27.809633    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 31 01:10:28.987782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:28.987744    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:28.988344 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:28.988323    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:28.989245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:28.989219    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:10:36.724854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.724812    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"]
Oct 31 01:10:36.729271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.729248    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.741242 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.741219    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-487a6c-openshift-default-dockercfg-fl6qs\""
Oct 31 01:10:36.744102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.744078    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"]
Oct 31 01:10:36.795478 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795445    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795478 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795488    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795713 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795513    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795713 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795691    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795753    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795807    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795927 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795909    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trtcm\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.795961 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.795952    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.796069 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.796055    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897208 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897177    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trtcm\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897226    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897292    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897323    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897354    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897395    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897526    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897662    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897743    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897867 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897846    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.897935 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.897909    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.898018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.898001    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.898112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.898090    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.898158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.898143    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.900117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.900091    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.900279 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.900263    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.905899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.905869    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:36.906092 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:36.906075    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trtcm\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:37.042867 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.042785    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:37.340827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.340803    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"]
Oct 31 01:10:37.343326 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:10:37.343302    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfae1a84_3356_4494_9064_43a79c66029f.slice/crio-d1501bbc4440175f65a9e8b84e9cbf89541ce3e55d826e6c1c35c37da6f75b71 WatchSource:0}: Error finding container d1501bbc4440175f65a9e8b84e9cbf89541ce3e55d826e6c1c35c37da6f75b71: Status 404 returned error can't find the container with id d1501bbc4440175f65a9e8b84e9cbf89541ce3e55d826e6c1c35c37da6f75b71
Oct 31 01:10:37.345842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.345808    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:37.345911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.345867    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:37.345911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.345898    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:10:37.439475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.439420    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" event={"ID":"cfae1a84-3356-4494-9064-43a79c66029f","Type":"ContainerStarted","Data":"65cf5c93705f670bb2ac60cf36463bce5fe6c94f0329cb009e90f9fb15c45783"}
Oct 31 01:10:37.439717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.439510    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" event={"ID":"cfae1a84-3356-4494-9064-43a79c66029f","Type":"ContainerStarted","Data":"d1501bbc4440175f65a9e8b84e9cbf89541ce3e55d826e6c1c35c37da6f75b71"}
Oct 31 01:10:37.481128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:37.481078    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podStartSLOduration=1.4810632830000001 podStartE2EDuration="1.481063283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:10:37.480079192 +0000 UTC m=+834.467191171" watchObservedRunningTime="2026-10-31 01:10:37.481063283 +0000 UTC m=+834.468175259"
Oct 31 01:10:38.043236 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:38.043201    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:38.044952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:38.044928    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 01:10:38.045059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:38.044977    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 01:10:39.047078 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:39.047036    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 01:10:39.047705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:39.047667    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 01:10:40.043347 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:40.043310    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 01:10:40.043521 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:40.043385    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 01:10:41.043545 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:41.043505    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 01:10:41.043994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:41.043567    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 01:10:42.043254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:42.043212    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 01:10:42.043435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:42.043283    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 01:10:43.043653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:43.043586    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 01:10:43.044047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:43.043668    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 01:10:44.315493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:44.315455    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:44.316204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:44.315881    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:44.316775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:44.316757    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:10:49.443071 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:10:49.443026    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:10:59.596946 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.596921    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"]
Oct 31 01:10:59.600822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.600805    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.603992 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.603970    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:10:59.604082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.604050    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:10:59.615109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.615086    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"]
Oct 31 01:10:59.669865 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.669836    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjphq\" (UniqueName: \"kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.670047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.670029    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.670091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.670076    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.704382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.704362    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:10:59.704382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.704373    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:10:59.771270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.771237    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.771449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.771301    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.771449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.771370    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjphq\" (UniqueName: \"kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.772066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.772045    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.773782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.773765    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.780140 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.780115    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjphq\" (UniqueName: \"kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq\") pod \"maas-api-revoke-sub-d951865f-4ftjf\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:10:59.914468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:10:59.914438    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:11:00.190454 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:00.190428    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"]
Oct 31 01:11:00.201444 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:00.198085    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0bca43_56d1_43fb_8abb_f1442f4abcbe.slice/crio-315f4529dcbb7a5e8fd82273eea0d3597179a33374b395952b3aaed15a033f9a WatchSource:0}: Error finding container 315f4529dcbb7a5e8fd82273eea0d3597179a33374b395952b3aaed15a033f9a: Status 404 returned error can't find the container with id 315f4529dcbb7a5e8fd82273eea0d3597179a33374b395952b3aaed15a033f9a
Oct 31 01:11:00.637121 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:00.637091    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf" event={"ID":"fd0bca43-56d1-43fb-8abb-f1442f4abcbe","Type":"ContainerStarted","Data":"315f4529dcbb7a5e8fd82273eea0d3597179a33374b395952b3aaed15a033f9a"}
Oct 31 01:11:01.815158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.815096    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"]
Oct 31 01:11:01.819745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.819705    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:01.834210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.834176    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"]
Oct 31 01:11:01.898682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.898639    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpfl\" (UniqueName: \"kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:01.898852 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.898751    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:01.898998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.898982    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:01.999843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.999809    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.000030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.999858    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpfl\" (UniqueName: \"kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.000030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:01.999899    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.000706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:02.000681    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.002468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:02.002443    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.008688 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:02.008664    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpfl\" (UniqueName: \"kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl\") pod \"maas-api-revoke-sub-eed1f1d4-gzljw\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.149528 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:02.149492    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:02.463635 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:02.463586    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2061e09_f874_45ad_9873_c5eb5e6ee96f.slice/crio-e9180e95dd948e336e970a6c4a77b9f440f4e5eb979e2cad55153a689e77f570 WatchSource:0}: Error finding container e9180e95dd948e336e970a6c4a77b9f440f4e5eb979e2cad55153a689e77f570: Status 404 returned error can't find the container with id e9180e95dd948e336e970a6c4a77b9f440f4e5eb979e2cad55153a689e77f570
Oct 31 01:11:02.463770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:02.463662    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"]
Oct 31 01:11:02.651197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:02.651157    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw" event={"ID":"f2061e09-f874-45ad-9873-c5eb5e6ee96f","Type":"ContainerStarted","Data":"e9180e95dd948e336e970a6c4a77b9f440f4e5eb979e2cad55153a689e77f570"}
Oct 31 01:11:03.657991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:03.657956    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf" event={"ID":"fd0bca43-56d1-43fb-8abb-f1442f4abcbe","Type":"ContainerStarted","Data":"38a32b0b1501ec375ace541c6f22a5fe68ad83a637e88075d76c789c33985174"}
Oct 31 01:11:03.674810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:03.674738    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf" podStartSLOduration=2.068602015 podStartE2EDuration="4.674725292s" totalImagesPullingTime="2.606123277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:03.67334343 +0000 UTC m=+860.660455406" watchObservedRunningTime="2026-10-31 01:11:03.674725292 +0000 UTC m=+860.661837268"
Oct 31 01:11:04.665711 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:04.665675    8029 generic.go:356] "Generic (PLEG): container finished" podID="f2061e09-f874-45ad-9873-c5eb5e6ee96f" containerID="ab9642f0fc5e60f89b57d6b2810aa2e86b589a4305f6aed12b9a90595360007c" exitCode=0
Oct 31 01:11:04.666146 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:04.665790    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw" event={"ID":"f2061e09-f874-45ad-9873-c5eb5e6ee96f","Type":"ContainerDied","Data":"ab9642f0fc5e60f89b57d6b2810aa2e86b589a4305f6aed12b9a90595360007c"}
Oct 31 01:11:04.667864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:04.667838    8029 generic.go:356] "Generic (PLEG): container finished" podID="fd0bca43-56d1-43fb-8abb-f1442f4abcbe" containerID="38a32b0b1501ec375ace541c6f22a5fe68ad83a637e88075d76c789c33985174" exitCode=0
Oct 31 01:11:04.667999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:04.667872    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf" event={"ID":"fd0bca43-56d1-43fb-8abb-f1442f4abcbe","Type":"ContainerDied","Data":"38a32b0b1501ec375ace541c6f22a5fe68ad83a637e88075d76c789c33985174"}
Oct 31 01:11:05.867180 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:05.867156    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:05.883240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:05.883213    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:11:06.056166 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056066    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca\") pod \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") "
Oct 31 01:11:06.056337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056186    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca\") pod \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") "
Oct 31 01:11:06.056401 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056352    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl\" (UniqueName: \"kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl\") pod \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") "
Oct 31 01:11:06.056457 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056401    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca\") pod \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") "
Oct 31 01:11:06.056457 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056433    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca\") pod \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\" (UID: \"f2061e09-f874-45ad-9873-c5eb5e6ee96f\") "
Oct 31 01:11:06.056557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056491    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq\" (UniqueName: \"kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq\") pod \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\" (UID: \"fd0bca43-56d1-43fb-8abb-f1442f4abcbe\") "
Oct 31 01:11:06.056877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056846    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca" pod "fd0bca43-56d1-43fb-8abb-f1442f4abcbe" (UID: "fd0bca43-56d1-43fb-8abb-f1442f4abcbe"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:06.056991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.056943    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca" pod "f2061e09-f874-45ad-9873-c5eb5e6ee96f" (UID: "f2061e09-f874-45ad-9873-c5eb5e6ee96f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:06.057049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.057009    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:06.059801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.059775    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq" pod "fd0bca43-56d1-43fb-8abb-f1442f4abcbe" (UID: "fd0bca43-56d1-43fb-8abb-f1442f4abcbe"). InnerVolumeSpecName "kube-api-access-tjphq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:06.061889 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.061410    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl" pod "f2061e09-f874-45ad-9873-c5eb5e6ee96f" (UID: "f2061e09-f874-45ad-9873-c5eb5e6ee96f"). InnerVolumeSpecName "kube-api-access-xnpfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:06.065197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.064966    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca" pod "f2061e09-f874-45ad-9873-c5eb5e6ee96f" (UID: "f2061e09-f874-45ad-9873-c5eb5e6ee96f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:06.065197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.065017    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca" pod "fd0bca43-56d1-43fb-8abb-f1442f4abcbe" (UID: "fd0bca43-56d1-43fb-8abb-f1442f4abcbe"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:06.159652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.158885    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjphq\" (UniqueName: \"kubernetes.io/projected/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-kube-api-access-tjphq\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:06.159652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.158922    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:06.159652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.158937    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd0bca43-56d1-43fb-8abb-f1442f4abcbe-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:06.159652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.158952    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnpfl\" (UniqueName: \"kubernetes.io/projected/f2061e09-f874-45ad-9873-c5eb5e6ee96f-kube-api-access-xnpfl\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:06.159652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.158966    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2061e09-f874-45ad-9873-c5eb5e6ee96f-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:06.690374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.689651    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw" event={"ID":"f2061e09-f874-45ad-9873-c5eb5e6ee96f","Type":"ContainerDied","Data":"e9180e95dd948e336e970a6c4a77b9f440f4e5eb979e2cad55153a689e77f570"}
Oct 31 01:11:06.690374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.689733    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9180e95dd948e336e970a6c4a77b9f440f4e5eb979e2cad55153a689e77f570"
Oct 31 01:11:06.690374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.689897    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"
Oct 31 01:11:06.691197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.691170    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf" event={"ID":"fd0bca43-56d1-43fb-8abb-f1442f4abcbe","Type":"ContainerDied","Data":"315f4529dcbb7a5e8fd82273eea0d3597179a33374b395952b3aaed15a033f9a"}
Oct 31 01:11:06.691335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.691213    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="315f4529dcbb7a5e8fd82273eea0d3597179a33374b395952b3aaed15a033f9a"
Oct 31 01:11:06.691335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:06.691218    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"
Oct 31 01:11:08.378086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378054    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"]
Oct 31 01:11:08.378653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378638    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fd0bca43-56d1-43fb-8abb-f1442f4abcbe" containerName="revoke-keys"
Oct 31 01:11:08.378653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378652    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd0bca43-56d1-43fb-8abb-f1442f4abcbe" containerName="revoke-keys"
Oct 31 01:11:08.378743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378669    8029 cpu_manager.go:414] "containerMap: removing container" podUID="f2061e09-f874-45ad-9873-c5eb5e6ee96f" containerName="revoke-keys"
Oct 31 01:11:08.378743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378674    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2061e09-f874-45ad-9873-c5eb5e6ee96f" containerName="revoke-keys"
Oct 31 01:11:08.378804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378751    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2061e09-f874-45ad-9873-c5eb5e6ee96f" containerName="revoke-keys"
Oct 31 01:11:08.378804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.378765    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd0bca43-56d1-43fb-8abb-f1442f4abcbe" containerName="revoke-keys"
Oct 31 01:11:08.382781 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.382759    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.385927 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.385902    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.386059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.386044    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.386112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.386074    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk8hp\" (UniqueName: \"kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.387903 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.387862    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:11:08.393022 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.388141    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:11:08.393111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.388199    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:11:08.393111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.389187    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:11:08.393111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.392161    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"]
Oct 31 01:11:08.487367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.487325    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.487554 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.487428    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.487554 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.487465    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk8hp\" (UniqueName: \"kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.488155 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.488115    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.490202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.490181    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.496708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.496679    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk8hp\" (UniqueName: \"kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp\") pod \"maas-api-revoke-sub-4e8b2b56-6jkdm\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:08.703723 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:08.703641    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:09.057522 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.057493    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"]
Oct 31 01:11:09.060905 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:09.060870    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5186862a_65f0_4650_82bc_999bbe3c6351.slice/crio-46adc8e1626618c6badd55867d3b271d2a2b0204cc184ce63868178e84689b7b WatchSource:0}: Error finding container 46adc8e1626618c6badd55867d3b271d2a2b0204cc184ce63868178e84689b7b: Status 404 returned error can't find the container with id 46adc8e1626618c6badd55867d3b271d2a2b0204cc184ce63868178e84689b7b
Oct 31 01:11:09.715819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.715732    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm" event={"ID":"5186862a-65f0-4650-82bc-999bbe3c6351","Type":"ContainerStarted","Data":"daeec6560bd5a25fa109e59def75fa07f7da3f9dbefc356957465cbc4996b511"}
Oct 31 01:11:09.715819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.715796    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm" event={"ID":"5186862a-65f0-4650-82bc-999bbe3c6351","Type":"ContainerStarted","Data":"46adc8e1626618c6badd55867d3b271d2a2b0204cc184ce63868178e84689b7b"}
Oct 31 01:11:09.723841 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.723813    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"]
Oct 31 01:11:09.727782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.727767    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.735579 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.735538    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"]
Oct 31 01:11:09.738258 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.738208    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm" podStartSLOduration=1.738191741 podStartE2EDuration="1.738191741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:09.736665054 +0000 UTC m=+866.723777029" watchObservedRunningTime="2026-10-31 01:11:09.738191741 +0000 UTC m=+866.725303719"
Oct 31 01:11:09.834464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.834433    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.834663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.834532    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqhb4\" (UniqueName: \"kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.834722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.834701    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.935676 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.935641    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.935860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.935764    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.935860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.935828    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqhb4\" (UniqueName: \"kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.936277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.936256    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.938552 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.938529    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:09.943999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:09.943974    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqhb4\" (UniqueName: \"kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4\") pod \"maas-api-revoke-sub-a4e58b98-6crzn\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:10.044225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.044146    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:10.339915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.339885    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"]
Oct 31 01:11:10.342504 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:10.342468    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafdd6c2_786d_48d7_98c3_4dbe96d40a99.slice/crio-ffd0df12f859ffe89d9f5d1f41f699d17b4157209e106a0fea55c9251f7acc23 WatchSource:0}: Error finding container ffd0df12f859ffe89d9f5d1f41f699d17b4157209e106a0fea55c9251f7acc23: Status 404 returned error can't find the container with id ffd0df12f859ffe89d9f5d1f41f699d17b4157209e106a0fea55c9251f7acc23
Oct 31 01:11:10.725124 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.725087    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn" event={"ID":"aafdd6c2-786d-48d7-98c3-4dbe96d40a99","Type":"ContainerStarted","Data":"24fbbb026f4caa1940012788ec27c95d4015e59f3f8f0dab501e5daca8b7ca0c"}
Oct 31 01:11:10.725644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.725149    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn" event={"ID":"aafdd6c2-786d-48d7-98c3-4dbe96d40a99","Type":"ContainerStarted","Data":"ffd0df12f859ffe89d9f5d1f41f699d17b4157209e106a0fea55c9251f7acc23"}
Oct 31 01:11:10.727096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.727066    8029 generic.go:356] "Generic (PLEG): container finished" podID="5186862a-65f0-4650-82bc-999bbe3c6351" containerID="daeec6560bd5a25fa109e59def75fa07f7da3f9dbefc356957465cbc4996b511" exitCode=0
Oct 31 01:11:10.727258 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.727239    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm" event={"ID":"5186862a-65f0-4650-82bc-999bbe3c6351","Type":"ContainerDied","Data":"daeec6560bd5a25fa109e59def75fa07f7da3f9dbefc356957465cbc4996b511"}
Oct 31 01:11:10.748386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:10.747192    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn" podStartSLOduration=1.7471696410000002 podStartE2EDuration="1.747169641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:10.740219927 +0000 UTC m=+867.727331904" watchObservedRunningTime="2026-10-31 01:11:10.747169641 +0000 UTC m=+867.734281644"
Oct 31 01:11:11.735860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.735826    8029 generic.go:356] "Generic (PLEG): container finished" podID="aafdd6c2-786d-48d7-98c3-4dbe96d40a99" containerID="24fbbb026f4caa1940012788ec27c95d4015e59f3f8f0dab501e5daca8b7ca0c" exitCode=0
Oct 31 01:11:11.736323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.735916    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn" event={"ID":"aafdd6c2-786d-48d7-98c3-4dbe96d40a99","Type":"ContainerDied","Data":"24fbbb026f4caa1940012788ec27c95d4015e59f3f8f0dab501e5daca8b7ca0c"}
Oct 31 01:11:11.754718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.754681    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"]
Oct 31 01:11:11.759161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.759140    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.767976 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.767950    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"]
Oct 31 01:11:11.862216 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.862182    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bc7b\" (UniqueName: \"kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.862448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.862280    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.862448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.862426    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.912190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.912167    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"]
Oct 31 01:11:11.916309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.916294    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:11.924052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.924030    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"]
Oct 31 01:11:11.925068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.925044    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:11.963895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.963869    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp\" (UniqueName: \"kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp\") pod \"5186862a-65f0-4650-82bc-999bbe3c6351\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") "
Oct 31 01:11:11.964078 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.963967    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca\") pod \"5186862a-65f0-4650-82bc-999bbe3c6351\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") "
Oct 31 01:11:11.964369 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.964169    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca\") pod \"5186862a-65f0-4650-82bc-999bbe3c6351\" (UID: \"5186862a-65f0-4650-82bc-999bbe3c6351\") "
Oct 31 01:11:11.964649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.964489    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bc7b\" (UniqueName: \"kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.964649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.964544    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:11.964797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.964654    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.964797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.964719    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca" pod "5186862a-65f0-4650-82bc-999bbe3c6351" (UID: "5186862a-65f0-4650-82bc-999bbe3c6351"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:11.964797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.964748    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79kbm\" (UniqueName: \"kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:11.965085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.965066    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.965170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.965100    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:11.965233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.965187    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:11.965611 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.965568    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.966341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.966320    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca" pod "5186862a-65f0-4650-82bc-999bbe3c6351" (UID: "5186862a-65f0-4650-82bc-999bbe3c6351"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:11.967847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.967827    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:11.967935 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.967896    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp" pod "5186862a-65f0-4650-82bc-999bbe3c6351" (UID: "5186862a-65f0-4650-82bc-999bbe3c6351"). InnerVolumeSpecName "kube-api-access-jk8hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:11.971922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:11.971897    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bc7b\" (UniqueName: \"kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b\") pod \"maas-api-revoke-sub-9f4eec7a-mf5xq\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:12.066672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.066566    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.066819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.066757    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79kbm\" (UniqueName: \"kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.066819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.066792    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.066931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.066918    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk8hp\" (UniqueName: \"kubernetes.io/projected/5186862a-65f0-4650-82bc-999bbe3c6351-kube-api-access-jk8hp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:12.066984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.066937    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5186862a-65f0-4650-82bc-999bbe3c6351-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:12.067287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.067262    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.069326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.069305    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.075510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.075490    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79kbm\" (UniqueName: \"kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm\") pod \"maas-api-revoke-sub-7075181b-g7sqh\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.080494 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.080475    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:12.239011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.238983    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:12.376039 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.375836    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"]
Oct 31 01:11:12.381142 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:12.381111    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaff83dd_8191_43d6_af5c_1a4f93116525.slice/crio-532b317aa66016ac30924b2cfa7fd7157f4dd37b8ee6a80e6e2180082c0b8916 WatchSource:0}: Error finding container 532b317aa66016ac30924b2cfa7fd7157f4dd37b8ee6a80e6e2180082c0b8916: Status 404 returned error can't find the container with id 532b317aa66016ac30924b2cfa7fd7157f4dd37b8ee6a80e6e2180082c0b8916
Oct 31 01:11:12.516134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.516111    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"]
Oct 31 01:11:12.518376 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:12.518347    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0e1084_caf2_4e4c_ad78_70519dc3d864.slice/crio-643793a8470e680c4f6e60bfd1649e3d05a5213896a9d3c0e1e0a32f6791c046 WatchSource:0}: Error finding container 643793a8470e680c4f6e60bfd1649e3d05a5213896a9d3c0e1e0a32f6791c046: Status 404 returned error can't find the container with id 643793a8470e680c4f6e60bfd1649e3d05a5213896a9d3c0e1e0a32f6791c046
Oct 31 01:11:12.742584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.742557    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh" event={"ID":"df0e1084-caf2-4e4c-ad78-70519dc3d864","Type":"ContainerStarted","Data":"643793a8470e680c4f6e60bfd1649e3d05a5213896a9d3c0e1e0a32f6791c046"}
Oct 31 01:11:12.744277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.744243    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq" event={"ID":"aaff83dd-8191-43d6-af5c-1a4f93116525","Type":"ContainerStarted","Data":"6fcd384f2c07894ce082468fdc3524dac8d62dc056d9a47065aca5b2770572b0"}
Oct 31 01:11:12.744399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.744298    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq" event={"ID":"aaff83dd-8191-43d6-af5c-1a4f93116525","Type":"ContainerStarted","Data":"532b317aa66016ac30924b2cfa7fd7157f4dd37b8ee6a80e6e2180082c0b8916"}
Oct 31 01:11:12.745571 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.745547    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"
Oct 31 01:11:12.745690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.745589    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm" event={"ID":"5186862a-65f0-4650-82bc-999bbe3c6351","Type":"ContainerDied","Data":"46adc8e1626618c6badd55867d3b271d2a2b0204cc184ce63868178e84689b7b"}
Oct 31 01:11:12.745690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.745642    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46adc8e1626618c6badd55867d3b271d2a2b0204cc184ce63868178e84689b7b"
Oct 31 01:11:12.761134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.761084    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq" podStartSLOduration=1.761070042 podStartE2EDuration="1.761070042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:12.75821788 +0000 UTC m=+869.745329859" watchObservedRunningTime="2026-10-31 01:11:12.761070042 +0000 UTC m=+869.748182018"
Oct 31 01:11:12.895214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.895185    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:12.978284 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.978196    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca\") pod \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") "
Oct 31 01:11:12.978459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.978289    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca\") pod \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") "
Oct 31 01:11:12.978459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.978326    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4\" (UniqueName: \"kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4\") pod \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\" (UID: \"aafdd6c2-786d-48d7-98c3-4dbe96d40a99\") "
Oct 31 01:11:12.978919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.978892    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca" pod "aafdd6c2-786d-48d7-98c3-4dbe96d40a99" (UID: "aafdd6c2-786d-48d7-98c3-4dbe96d40a99"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:12.980885 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.980856    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4" pod "aafdd6c2-786d-48d7-98c3-4dbe96d40a99" (UID: "aafdd6c2-786d-48d7-98c3-4dbe96d40a99"). InnerVolumeSpecName "kube-api-access-dqhb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:12.981497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:12.981464    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca" pod "aafdd6c2-786d-48d7-98c3-4dbe96d40a99" (UID: "aafdd6c2-786d-48d7-98c3-4dbe96d40a99"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:13.079655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.079617    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:13.079655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.079643    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:13.079655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.079654    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqhb4\" (UniqueName: \"kubernetes.io/projected/aafdd6c2-786d-48d7-98c3-4dbe96d40a99-kube-api-access-dqhb4\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:13.752886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.752854    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn" event={"ID":"aafdd6c2-786d-48d7-98c3-4dbe96d40a99","Type":"ContainerDied","Data":"ffd0df12f859ffe89d9f5d1f41f699d17b4157209e106a0fea55c9251f7acc23"}
Oct 31 01:11:13.752886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.752872    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"
Oct 31 01:11:13.753350 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.752935    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffd0df12f859ffe89d9f5d1f41f699d17b4157209e106a0fea55c9251f7acc23"
Oct 31 01:11:13.754448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.754423    8029 generic.go:356] "Generic (PLEG): container finished" podID="aaff83dd-8191-43d6-af5c-1a4f93116525" containerID="6fcd384f2c07894ce082468fdc3524dac8d62dc056d9a47065aca5b2770572b0" exitCode=0
Oct 31 01:11:13.754588 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.754507    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq" event={"ID":"aaff83dd-8191-43d6-af5c-1a4f93116525","Type":"ContainerDied","Data":"6fcd384f2c07894ce082468fdc3524dac8d62dc056d9a47065aca5b2770572b0"}
Oct 31 01:11:13.756301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.756275    8029 generic.go:356] "Generic (PLEG): container finished" podID="df0e1084-caf2-4e4c-ad78-70519dc3d864" containerID="c3311add1d9319c88eff8a5435ccefe21b21f7edaf64688c06e3139121550f0a" exitCode=0
Oct 31 01:11:13.756464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:13.756335    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh" event={"ID":"df0e1084-caf2-4e4c-ad78-70519dc3d864","Type":"ContainerDied","Data":"c3311add1d9319c88eff8a5435ccefe21b21f7edaf64688c06e3139121550f0a"}
Oct 31 01:11:14.981151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:14.981125    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:14.983163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:14.983142    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:15.103792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.103766    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca\") pod \"df0e1084-caf2-4e4c-ad78-70519dc3d864\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") "
Oct 31 01:11:15.103960 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.103864    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca\") pod \"df0e1084-caf2-4e4c-ad78-70519dc3d864\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") "
Oct 31 01:11:15.103960 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.103913    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b\" (UniqueName: \"kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b\") pod \"aaff83dd-8191-43d6-af5c-1a4f93116525\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") "
Oct 31 01:11:15.104082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.103967    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm\" (UniqueName: \"kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm\") pod \"df0e1084-caf2-4e4c-ad78-70519dc3d864\" (UID: \"df0e1084-caf2-4e4c-ad78-70519dc3d864\") "
Oct 31 01:11:15.104082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.103991    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca\") pod \"aaff83dd-8191-43d6-af5c-1a4f93116525\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") "
Oct 31 01:11:15.104082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.104020    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca\") pod \"aaff83dd-8191-43d6-af5c-1a4f93116525\" (UID: \"aaff83dd-8191-43d6-af5c-1a4f93116525\") "
Oct 31 01:11:15.104244 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.104139    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca" pod "df0e1084-caf2-4e4c-ad78-70519dc3d864" (UID: "df0e1084-caf2-4e4c-ad78-70519dc3d864"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:15.104617 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.104567    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca" pod "aaff83dd-8191-43d6-af5c-1a4f93116525" (UID: "aaff83dd-8191-43d6-af5c-1a4f93116525"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:15.104908 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.104640    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:15.106353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.106326    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm" pod "df0e1084-caf2-4e4c-ad78-70519dc3d864" (UID: "df0e1084-caf2-4e4c-ad78-70519dc3d864"). InnerVolumeSpecName "kube-api-access-79kbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:15.106570 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.106541    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b" pod "aaff83dd-8191-43d6-af5c-1a4f93116525" (UID: "aaff83dd-8191-43d6-af5c-1a4f93116525"). InnerVolumeSpecName "kube-api-access-8bc7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:15.106756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.106739    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca" pod "df0e1084-caf2-4e4c-ad78-70519dc3d864" (UID: "df0e1084-caf2-4e4c-ad78-70519dc3d864"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:15.106806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.106750    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca" pod "aaff83dd-8191-43d6-af5c-1a4f93116525" (UID: "aaff83dd-8191-43d6-af5c-1a4f93116525"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:15.189337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.189304    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"]
Oct 31 01:11:15.190112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190095    8029 cpu_manager.go:414] "containerMap: removing container" podUID="aaff83dd-8191-43d6-af5c-1a4f93116525" containerName="revoke-keys"
Oct 31 01:11:15.190167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190115    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaff83dd-8191-43d6-af5c-1a4f93116525" containerName="revoke-keys"
Oct 31 01:11:15.190167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190145    8029 cpu_manager.go:414] "containerMap: removing container" podUID="df0e1084-caf2-4e4c-ad78-70519dc3d864" containerName="revoke-keys"
Oct 31 01:11:15.190167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190152    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df0e1084-caf2-4e4c-ad78-70519dc3d864" containerName="revoke-keys"
Oct 31 01:11:15.190288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190178    8029 cpu_manager.go:414] "containerMap: removing container" podUID="aafdd6c2-786d-48d7-98c3-4dbe96d40a99" containerName="revoke-keys"
Oct 31 01:11:15.190288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190187    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafdd6c2-786d-48d7-98c3-4dbe96d40a99" containerName="revoke-keys"
Oct 31 01:11:15.190288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190203    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5186862a-65f0-4650-82bc-999bbe3c6351" containerName="revoke-keys"
Oct 31 01:11:15.190288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190210    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5186862a-65f0-4650-82bc-999bbe3c6351" containerName="revoke-keys"
Oct 31 01:11:15.190751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190473    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="df0e1084-caf2-4e4c-ad78-70519dc3d864" containerName="revoke-keys"
Oct 31 01:11:15.190751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190493    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafdd6c2-786d-48d7-98c3-4dbe96d40a99" containerName="revoke-keys"
Oct 31 01:11:15.190751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190504    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5186862a-65f0-4650-82bc-999bbe3c6351" containerName="revoke-keys"
Oct 31 01:11:15.190751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.190524    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaff83dd-8191-43d6-af5c-1a4f93116525" containerName="revoke-keys"
Oct 31 01:11:15.196855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.196833    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.210840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.206140    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79kbm\" (UniqueName: \"kubernetes.io/projected/df0e1084-caf2-4e4c-ad78-70519dc3d864-kube-api-access-79kbm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:15.210840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.206162    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:15.210840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.206173    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aaff83dd-8191-43d6-af5c-1a4f93116525-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:15.210840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.206286    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0e1084-caf2-4e4c-ad78-70519dc3d864-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:15.210840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.206300    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bc7b\" (UniqueName: \"kubernetes.io/projected/aaff83dd-8191-43d6-af5c-1a4f93116525-kube-api-access-8bc7b\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:15.210840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.207339    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"]
Oct 31 01:11:15.307654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.307622    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.307846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.307755    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.307846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.307819    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hnhc\" (UniqueName: \"kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.408909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.408817    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.409063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.408944    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.409063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.409004    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hnhc\" (UniqueName: \"kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.409660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.409627    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.411379 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.411354    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.416785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.416761    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hnhc\" (UniqueName: \"kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc\") pod \"maas-api-revoke-sub-4e43c771-22zh9\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.526881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.526851    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:15.769884 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.769816    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"
Oct 31 01:11:15.769884 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.769849    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh" event={"ID":"df0e1084-caf2-4e4c-ad78-70519dc3d864","Type":"ContainerDied","Data":"643793a8470e680c4f6e60bfd1649e3d05a5213896a9d3c0e1e0a32f6791c046"}
Oct 31 01:11:15.770086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.769903    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="643793a8470e680c4f6e60bfd1649e3d05a5213896a9d3c0e1e0a32f6791c046"
Oct 31 01:11:15.771308 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.771291    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"
Oct 31 01:11:15.771308 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.771304    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq" event={"ID":"aaff83dd-8191-43d6-af5c-1a4f93116525","Type":"ContainerDied","Data":"532b317aa66016ac30924b2cfa7fd7157f4dd37b8ee6a80e6e2180082c0b8916"}
Oct 31 01:11:15.771450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.771333    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="532b317aa66016ac30924b2cfa7fd7157f4dd37b8ee6a80e6e2180082c0b8916"
Oct 31 01:11:15.801653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:15.801629    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"]
Oct 31 01:11:15.803465 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:15.803441    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bcb1430_31ce_4de6_929b_237879a0a46b.slice/crio-19b47f4d292d06f74e006c88ac6480733e26cdedc070172ad23766d9585446ee WatchSource:0}: Error finding container 19b47f4d292d06f74e006c88ac6480733e26cdedc070172ad23766d9585446ee: Status 404 returned error can't find the container with id 19b47f4d292d06f74e006c88ac6480733e26cdedc070172ad23766d9585446ee
Oct 31 01:11:16.778977 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:16.778945    8029 generic.go:356] "Generic (PLEG): container finished" podID="1bcb1430-31ce-4de6-929b-237879a0a46b" containerID="1daaa26379668dbb1410061a71fa9110317175686b7694c69f7b2291bc63f64b" exitCode=0
Oct 31 01:11:16.779327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:16.779023    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9" event={"ID":"1bcb1430-31ce-4de6-929b-237879a0a46b","Type":"ContainerDied","Data":"1daaa26379668dbb1410061a71fa9110317175686b7694c69f7b2291bc63f64b"}
Oct 31 01:11:16.779327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:16.779092    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9" event={"ID":"1bcb1430-31ce-4de6-929b-237879a0a46b","Type":"ContainerStarted","Data":"19b47f4d292d06f74e006c88ac6480733e26cdedc070172ad23766d9585446ee"}
Oct 31 01:11:18.281854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.281822    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:18.366675 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.366649    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca\") pod \"1bcb1430-31ce-4de6-929b-237879a0a46b\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") "
Oct 31 01:11:18.366838 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.366688    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc\" (UniqueName: \"kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc\") pod \"1bcb1430-31ce-4de6-929b-237879a0a46b\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") "
Oct 31 01:11:18.366838 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.366810    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca\") pod \"1bcb1430-31ce-4de6-929b-237879a0a46b\" (UID: \"1bcb1430-31ce-4de6-929b-237879a0a46b\") "
Oct 31 01:11:18.367149 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.367124    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca" pod "1bcb1430-31ce-4de6-929b-237879a0a46b" (UID: "1bcb1430-31ce-4de6-929b-237879a0a46b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:18.368860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.368839    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc" pod "1bcb1430-31ce-4de6-929b-237879a0a46b" (UID: "1bcb1430-31ce-4de6-929b-237879a0a46b"). InnerVolumeSpecName "kube-api-access-8hnhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:18.368955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.368886    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca" pod "1bcb1430-31ce-4de6-929b-237879a0a46b" (UID: "1bcb1430-31ce-4de6-929b-237879a0a46b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:18.467909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.467875    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:18.467909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.467904    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bcb1430-31ce-4de6-929b-237879a0a46b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:18.467909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.467915    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hnhc\" (UniqueName: \"kubernetes.io/projected/1bcb1430-31ce-4de6-929b-237879a0a46b-kube-api-access-8hnhc\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:18.837087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.836997    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"
Oct 31 01:11:18.837087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.837019    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9" event={"ID":"1bcb1430-31ce-4de6-929b-237879a0a46b","Type":"ContainerDied","Data":"19b47f4d292d06f74e006c88ac6480733e26cdedc070172ad23766d9585446ee"}
Oct 31 01:11:18.837087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:18.837070    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19b47f4d292d06f74e006c88ac6480733e26cdedc070172ad23766d9585446ee"
Oct 31 01:11:19.447761 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:11:19.447727    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:11:19.779485 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.779404    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"]
Oct 31 01:11:19.779984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.779970    8029 cpu_manager.go:414] "containerMap: removing container" podUID="1bcb1430-31ce-4de6-929b-237879a0a46b" containerName="revoke-keys"
Oct 31 01:11:19.780041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.779985    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bcb1430-31ce-4de6-929b-237879a0a46b" containerName="revoke-keys"
Oct 31 01:11:19.780102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.780084    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bcb1430-31ce-4de6-929b-237879a0a46b" containerName="revoke-keys"
Oct 31 01:11:19.783710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.783690    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.788243 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.788225    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:11:19.788374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.788276    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:11:19.788374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.788305    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:11:19.788374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.788307    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:11:19.794362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.794072    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"]
Oct 31 01:11:19.886876 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.886846    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.887062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.886989    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bg9s\" (UniqueName: \"kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.887062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.887053    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.988433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.988396    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.988629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.988490    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.988629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.988593    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bg9s\" (UniqueName: \"kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.989197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.989168    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.990931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.990911    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:19.996443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:19.996421    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bg9s\" (UniqueName: \"kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s\") pod \"maas-api-revoke-sub-feb4ac6b-sfnww\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:20.098770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.098695    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:20.112411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.112387    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"]
Oct 31 01:11:20.116526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.116504    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.125572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.125538    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"]
Oct 31 01:11:20.190591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.190558    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.190756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.190662    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.190805 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.190786    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkw6q\" (UniqueName: \"kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.292738 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.292679    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.292964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.292775    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.293397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.293367    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkw6q\" (UniqueName: \"kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.294065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.294039    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.296887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.296848    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.315994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.315967    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkw6q\" (UniqueName: \"kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q\") pod \"maas-api-revoke-sub-f347c451-885rr\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.390683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.390656    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"]
Oct 31 01:11:20.390989 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:20.390963    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod604a805b_8913_4f7b_a750_b9e1a6d2a36c.slice/crio-2568f4672e28b323ef60ebf5b97ab96d12d26e9242add39f83ae7de05e68b7a5 WatchSource:0}: Error finding container 2568f4672e28b323ef60ebf5b97ab96d12d26e9242add39f83ae7de05e68b7a5: Status 404 returned error can't find the container with id 2568f4672e28b323ef60ebf5b97ab96d12d26e9242add39f83ae7de05e68b7a5
Oct 31 01:11:20.454627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.454569    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:20.735700 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.735676    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"]
Oct 31 01:11:20.737464 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:20.737444    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879be096_2fc1_4724_9702_feaa32fb8323.slice/crio-cdddfc208aa9b6b26324a870ca3b244c3b5156a040a86dca3ea3bf6519ba213d WatchSource:0}: Error finding container cdddfc208aa9b6b26324a870ca3b244c3b5156a040a86dca3ea3bf6519ba213d: Status 404 returned error can't find the container with id cdddfc208aa9b6b26324a870ca3b244c3b5156a040a86dca3ea3bf6519ba213d
Oct 31 01:11:20.853415 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.853364    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww" event={"ID":"604a805b-8913-4f7b-a750-b9e1a6d2a36c","Type":"ContainerStarted","Data":"c2923a4af0a96b47ba8bd69f1182c6b962b8a8289ff0d6755329b9ab86eae232"}
Oct 31 01:11:20.853591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.853437    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww" event={"ID":"604a805b-8913-4f7b-a750-b9e1a6d2a36c","Type":"ContainerStarted","Data":"2568f4672e28b323ef60ebf5b97ab96d12d26e9242add39f83ae7de05e68b7a5"}
Oct 31 01:11:20.854808 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:20.854784    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr" event={"ID":"879be096-2fc1-4724-9702-feaa32fb8323","Type":"ContainerStarted","Data":"cdddfc208aa9b6b26324a870ca3b244c3b5156a040a86dca3ea3bf6519ba213d"}
Oct 31 01:11:21.864912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:21.864879    8029 generic.go:356] "Generic (PLEG): container finished" podID="604a805b-8913-4f7b-a750-b9e1a6d2a36c" containerID="c2923a4af0a96b47ba8bd69f1182c6b962b8a8289ff0d6755329b9ab86eae232" exitCode=0
Oct 31 01:11:21.865339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:21.864955    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww" event={"ID":"604a805b-8913-4f7b-a750-b9e1a6d2a36c","Type":"ContainerDied","Data":"c2923a4af0a96b47ba8bd69f1182c6b962b8a8289ff0d6755329b9ab86eae232"}
Oct 31 01:11:21.866587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:21.866567    8029 generic.go:356] "Generic (PLEG): container finished" podID="879be096-2fc1-4724-9702-feaa32fb8323" containerID="8148d56b7e85285b1d87fb89809223f946c797cfdbc934c81a0a65ce40442132" exitCode=0
Oct 31 01:11:21.866705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:21.866637    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr" event={"ID":"879be096-2fc1-4724-9702-feaa32fb8323","Type":"ContainerDied","Data":"8148d56b7e85285b1d87fb89809223f946c797cfdbc934c81a0a65ce40442132"}
Oct 31 01:11:22.282857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.282782    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"]
Oct 31 01:11:22.286906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.286886    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.297318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.297292    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"]
Oct 31 01:11:22.419776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.419743    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.419776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.419778    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.420037 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.419960    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9qcx\" (UniqueName: \"kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.521366 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.521326    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9qcx\" (UniqueName: \"kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.521583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.521520    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.521583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.521552    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.522358 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.522320    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.524143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.524124    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.530255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.530230    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9qcx\" (UniqueName: \"kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx\") pod \"maas-api-revoke-sub-626021dd-jvl4g\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.602248 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.602228    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:22.921085 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:22.921047    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d959f49_7094_44a3_84ca_1f236606c356.slice/crio-045e97373e7c2db8b9b22d162283a33681bbd090152f6e1a83bd7f7f80de06d3 WatchSource:0}: Error finding container 045e97373e7c2db8b9b22d162283a33681bbd090152f6e1a83bd7f7f80de06d3: Status 404 returned error can't find the container with id 045e97373e7c2db8b9b22d162283a33681bbd090152f6e1a83bd7f7f80de06d3
Oct 31 01:11:22.924365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:22.924319    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"]
Oct 31 01:11:23.038857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.038832    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:23.104179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.104153    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:23.131504 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.131407    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q\" (UniqueName: \"kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q\") pod \"879be096-2fc1-4724-9702-feaa32fb8323\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") "
Oct 31 01:11:23.131682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.131594    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca\") pod \"879be096-2fc1-4724-9702-feaa32fb8323\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") "
Oct 31 01:11:23.131771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.131754    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca\") pod \"879be096-2fc1-4724-9702-feaa32fb8323\" (UID: \"879be096-2fc1-4724-9702-feaa32fb8323\") "
Oct 31 01:11:23.132570 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.132543    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca" pod "879be096-2fc1-4724-9702-feaa32fb8323" (UID: "879be096-2fc1-4724-9702-feaa32fb8323"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:23.135100 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.134980    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q" pod "879be096-2fc1-4724-9702-feaa32fb8323" (UID: "879be096-2fc1-4724-9702-feaa32fb8323"). InnerVolumeSpecName "kube-api-access-xkw6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:23.144804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.144779    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca" pod "879be096-2fc1-4724-9702-feaa32fb8323" (UID: "879be096-2fc1-4724-9702-feaa32fb8323"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:23.233154 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233129    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s\" (UniqueName: \"kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s\") pod \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") "
Oct 31 01:11:23.233282 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233242    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca\") pod \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") "
Oct 31 01:11:23.233353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233341    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca\") pod \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\" (UID: \"604a805b-8913-4f7b-a750-b9e1a6d2a36c\") "
Oct 31 01:11:23.233792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233753    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca" pod "604a805b-8913-4f7b-a750-b9e1a6d2a36c" (UID: "604a805b-8913-4f7b-a750-b9e1a6d2a36c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:23.233894 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233828    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:23.233894 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233846    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/879be096-2fc1-4724-9702-feaa32fb8323-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:23.233894 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233860    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:23.233894 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.233869    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkw6q\" (UniqueName: \"kubernetes.io/projected/879be096-2fc1-4724-9702-feaa32fb8323-kube-api-access-xkw6q\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:23.277619 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.277574    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca" pod "604a805b-8913-4f7b-a750-b9e1a6d2a36c" (UID: "604a805b-8913-4f7b-a750-b9e1a6d2a36c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:23.277768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.277694    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s" pod "604a805b-8913-4f7b-a750-b9e1a6d2a36c" (UID: "604a805b-8913-4f7b-a750-b9e1a6d2a36c"). InnerVolumeSpecName "kube-api-access-5bg9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:23.334853 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.334824    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/604a805b-8913-4f7b-a750-b9e1a6d2a36c-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:23.334853 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.334855    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bg9s\" (UniqueName: \"kubernetes.io/projected/604a805b-8913-4f7b-a750-b9e1a6d2a36c-kube-api-access-5bg9s\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:23.880917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.880884    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"
Oct 31 01:11:23.880917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.880904    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww" event={"ID":"604a805b-8913-4f7b-a750-b9e1a6d2a36c","Type":"ContainerDied","Data":"2568f4672e28b323ef60ebf5b97ab96d12d26e9242add39f83ae7de05e68b7a5"}
Oct 31 01:11:23.881096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.880953    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2568f4672e28b323ef60ebf5b97ab96d12d26e9242add39f83ae7de05e68b7a5"
Oct 31 01:11:23.882567 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.882538    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr" event={"ID":"879be096-2fc1-4724-9702-feaa32fb8323","Type":"ContainerDied","Data":"cdddfc208aa9b6b26324a870ca3b244c3b5156a040a86dca3ea3bf6519ba213d"}
Oct 31 01:11:23.882567 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.882564    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"
Oct 31 01:11:23.882778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.882593    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdddfc208aa9b6b26324a870ca3b244c3b5156a040a86dca3ea3bf6519ba213d"
Oct 31 01:11:23.886056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.886033    8029 generic.go:356] "Generic (PLEG): container finished" podID="7d959f49-7094-44a3-84ca-1f236606c356" containerID="2123a824ff6510b9c11ee4c3fe5eca891fe4df8d7949d8d26f2c21ba12858f44" exitCode=0
Oct 31 01:11:23.886229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.886212    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g" event={"ID":"7d959f49-7094-44a3-84ca-1f236606c356","Type":"ContainerDied","Data":"2123a824ff6510b9c11ee4c3fe5eca891fe4df8d7949d8d26f2c21ba12858f44"}
Oct 31 01:11:23.886332 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:23.886273    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g" event={"ID":"7d959f49-7094-44a3-84ca-1f236606c356","Type":"ContainerStarted","Data":"045e97373e7c2db8b9b22d162283a33681bbd090152f6e1a83bd7f7f80de06d3"}
Oct 31 01:11:24.484432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.484394    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"]
Oct 31 01:11:24.485191 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.485167    8029 cpu_manager.go:414] "containerMap: removing container" podUID="604a805b-8913-4f7b-a750-b9e1a6d2a36c" containerName="revoke-keys"
Oct 31 01:11:24.485290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.485193    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="604a805b-8913-4f7b-a750-b9e1a6d2a36c" containerName="revoke-keys"
Oct 31 01:11:24.485290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.485253    8029 cpu_manager.go:414] "containerMap: removing container" podUID="879be096-2fc1-4724-9702-feaa32fb8323" containerName="revoke-keys"
Oct 31 01:11:24.485290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.485261    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="879be096-2fc1-4724-9702-feaa32fb8323" containerName="revoke-keys"
Oct 31 01:11:24.485432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.485410    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="879be096-2fc1-4724-9702-feaa32fb8323" containerName="revoke-keys"
Oct 31 01:11:24.485432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.485429    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="604a805b-8913-4f7b-a750-b9e1a6d2a36c" containerName="revoke-keys"
Oct 31 01:11:24.488806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.488784    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.497330 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.497310    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"]
Oct 31 01:11:24.661807 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.661771    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.662012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.661843    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.662012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.661976    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqkr7\" (UniqueName: \"kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.763807 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.763726    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqkr7\" (UniqueName: \"kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.763956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.763854    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.763956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.763899    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.764500 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.764478    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.766385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.766352    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.771576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.771552    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqkr7\" (UniqueName: \"kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7\") pod \"maas-api-revoke-sub-8a73b0b2-g2cn4\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.807573 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.807549    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:24.961272 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.961244    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"]
Oct 31 01:11:24.967847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.967823    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:24.973780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:24.973711    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"]
Oct 31 01:11:25.067435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.067393    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:25.071392 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.068428    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhf7b\" (UniqueName: \"kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.071392 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.068504    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.071392 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.068820    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.107242 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.106129    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"]
Oct 31 01:11:25.169243 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169218    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx\" (UniqueName: \"kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx\") pod \"7d959f49-7094-44a3-84ca-1f236606c356\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") "
Oct 31 01:11:25.169371 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169288    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca\") pod \"7d959f49-7094-44a3-84ca-1f236606c356\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") "
Oct 31 01:11:25.169483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169470    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca\") pod \"7d959f49-7094-44a3-84ca-1f236606c356\" (UID: \"7d959f49-7094-44a3-84ca-1f236606c356\") "
Oct 31 01:11:25.169636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169617    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.169690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169644    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca" pod "7d959f49-7094-44a3-84ca-1f236606c356" (UID: "7d959f49-7094-44a3-84ca-1f236606c356"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:25.169821 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169803    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.169955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.169939    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhf7b\" (UniqueName: \"kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.170039 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.170028    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:25.170775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.170750    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.172456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.171750    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca" pod "7d959f49-7094-44a3-84ca-1f236606c356" (UID: "7d959f49-7094-44a3-84ca-1f236606c356"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:25.172456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.171849    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx" pod "7d959f49-7094-44a3-84ca-1f236606c356" (UID: "7d959f49-7094-44a3-84ca-1f236606c356"). InnerVolumeSpecName "kube-api-access-g9qcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:25.172456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.172078    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.178818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.178792    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhf7b\" (UniqueName: \"kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b\") pod \"maas-api-revoke-sub-9ed8f450-h82v7\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.271409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.271375    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d959f49-7094-44a3-84ca-1f236606c356-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:25.271409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.271411    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9qcx\" (UniqueName: \"kubernetes.io/projected/7d959f49-7094-44a3-84ca-1f236606c356-kube-api-access-g9qcx\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:25.294454 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.294429    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:25.571333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.571306    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"]
Oct 31 01:11:25.573015 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:25.572992    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a53a862_9a21_4196_9011_270d98389276.slice/crio-ba68bf764fdc76252310ce6b65e3a3a08c95cd95cc033e57b40f632b2400f4c7 WatchSource:0}: Error finding container ba68bf764fdc76252310ce6b65e3a3a08c95cd95cc033e57b40f632b2400f4c7: Status 404 returned error can't find the container with id ba68bf764fdc76252310ce6b65e3a3a08c95cd95cc033e57b40f632b2400f4c7
Oct 31 01:11:25.609589 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.609561    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"]
Oct 31 01:11:25.610203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.610186    8029 cpu_manager.go:414] "containerMap: removing container" podUID="7d959f49-7094-44a3-84ca-1f236606c356" containerName="revoke-keys"
Oct 31 01:11:25.610298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.610205    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d959f49-7094-44a3-84ca-1f236606c356" containerName="revoke-keys"
Oct 31 01:11:25.610383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.610370    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d959f49-7094-44a3-84ca-1f236606c356" containerName="revoke-keys"
Oct 31 01:11:25.615187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.615167    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.622498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.622476    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"]
Oct 31 01:11:25.778507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.778475    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz699\" (UniqueName: \"kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.778676 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.778543    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.778744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.778721    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.880531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.880500    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.880784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.880697    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz699\" (UniqueName: \"kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.880866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.880787    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.881259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.881235    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.883542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.883519    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.889037 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.889012    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz699\" (UniqueName: \"kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699\") pod \"maas-api-revoke-sub-c70795f8-kpx4s\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:25.903053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.903032    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"
Oct 31 01:11:25.903219 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.903055    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g" event={"ID":"7d959f49-7094-44a3-84ca-1f236606c356","Type":"ContainerDied","Data":"045e97373e7c2db8b9b22d162283a33681bbd090152f6e1a83bd7f7f80de06d3"}
Oct 31 01:11:25.903338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.903285    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="045e97373e7c2db8b9b22d162283a33681bbd090152f6e1a83bd7f7f80de06d3"
Oct 31 01:11:25.904535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.904510    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7" event={"ID":"2a53a862-9a21-4196-9011-270d98389276","Type":"ContainerStarted","Data":"ba68bf764fdc76252310ce6b65e3a3a08c95cd95cc033e57b40f632b2400f4c7"}
Oct 31 01:11:25.906148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.906129    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4" event={"ID":"992b3425-4206-49cf-8342-213c251d0504","Type":"ContainerStarted","Data":"75b2f21d4f0f2de15d4bbdfc7fc286d154ed00ed42dab250692586d0bb6d505f"}
Oct 31 01:11:25.906257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.906163    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4" event={"ID":"992b3425-4206-49cf-8342-213c251d0504","Type":"ContainerStarted","Data":"4016f32b775caff9b1d6cf2c1439fc7ee4f3fdb82509a38b58f0837b623e5059"}
Oct 31 01:11:25.924309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.924271    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4" podStartSLOduration=1.9242612289999999 podStartE2EDuration="1.924261229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:25.920864656 +0000 UTC m=+882.907976644" watchObservedRunningTime="2026-10-31 01:11:25.924261229 +0000 UTC m=+882.911373214"
Oct 31 01:11:25.934949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:25.934926    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:26.217566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.217543    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"]
Oct 31 01:11:26.218442 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:26.218417    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a18cee_2322_4250_8f46_84225409131b.slice/crio-5c9aa5818316239ae9ed0eaadfb4af8aec93f07cde24a15ffb97237fb7b98b04 WatchSource:0}: Error finding container 5c9aa5818316239ae9ed0eaadfb4af8aec93f07cde24a15ffb97237fb7b98b04: Status 404 returned error can't find the container with id 5c9aa5818316239ae9ed0eaadfb4af8aec93f07cde24a15ffb97237fb7b98b04
Oct 31 01:11:26.914223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.914187    8029 generic.go:356] "Generic (PLEG): container finished" podID="992b3425-4206-49cf-8342-213c251d0504" containerID="75b2f21d4f0f2de15d4bbdfc7fc286d154ed00ed42dab250692586d0bb6d505f" exitCode=0
Oct 31 01:11:26.914680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.914280    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4" event={"ID":"992b3425-4206-49cf-8342-213c251d0504","Type":"ContainerDied","Data":"75b2f21d4f0f2de15d4bbdfc7fc286d154ed00ed42dab250692586d0bb6d505f"}
Oct 31 01:11:26.915843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.915818    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s" event={"ID":"f5a18cee-2322-4250-8f46-84225409131b","Type":"ContainerStarted","Data":"e35004411a0507b3feff3b8e14b8161f95f5efc7e42f5b2a26e1fea7f2dc81ca"}
Oct 31 01:11:26.915919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.915866    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s" event={"ID":"f5a18cee-2322-4250-8f46-84225409131b","Type":"ContainerStarted","Data":"5c9aa5818316239ae9ed0eaadfb4af8aec93f07cde24a15ffb97237fb7b98b04"}
Oct 31 01:11:26.917358 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.917333    8029 generic.go:356] "Generic (PLEG): container finished" podID="2a53a862-9a21-4196-9011-270d98389276" containerID="c5aab19dc400757d340f03643a1286f742a6af8e4226a542d0b4aedd5e36d576" exitCode=0
Oct 31 01:11:26.917449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.917412    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7" event={"ID":"2a53a862-9a21-4196-9011-270d98389276","Type":"ContainerDied","Data":"c5aab19dc400757d340f03643a1286f742a6af8e4226a542d0b4aedd5e36d576"}
Oct 31 01:11:26.950852 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:26.950803    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s" podStartSLOduration=1.950786056 podStartE2EDuration="1.950786056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:26.950006978 +0000 UTC m=+883.937118955" watchObservedRunningTime="2026-10-31 01:11:26.950786056 +0000 UTC m=+883.937898035"
Oct 31 01:11:27.926554 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:27.926523    8029 generic.go:356] "Generic (PLEG): container finished" podID="f5a18cee-2322-4250-8f46-84225409131b" containerID="e35004411a0507b3feff3b8e14b8161f95f5efc7e42f5b2a26e1fea7f2dc81ca" exitCode=0
Oct 31 01:11:27.927020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:27.926590    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s" event={"ID":"f5a18cee-2322-4250-8f46-84225409131b","Type":"ContainerDied","Data":"e35004411a0507b3feff3b8e14b8161f95f5efc7e42f5b2a26e1fea7f2dc81ca"}
Oct 31 01:11:28.125376 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.125352    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:28.156943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.156917    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:28.307645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307550    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca\") pod \"2a53a862-9a21-4196-9011-270d98389276\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") "
Oct 31 01:11:28.307784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307699    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca\") pod \"992b3425-4206-49cf-8342-213c251d0504\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") "
Oct 31 01:11:28.307784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307767    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b\" (UniqueName: \"kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b\") pod \"2a53a862-9a21-4196-9011-270d98389276\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") "
Oct 31 01:11:28.307904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307799    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca\") pod \"2a53a862-9a21-4196-9011-270d98389276\" (UID: \"2a53a862-9a21-4196-9011-270d98389276\") "
Oct 31 01:11:28.307904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307835    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7\" (UniqueName: \"kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7\") pod \"992b3425-4206-49cf-8342-213c251d0504\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") "
Oct 31 01:11:28.307904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307892    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca\") pod \"992b3425-4206-49cf-8342-213c251d0504\" (UID: \"992b3425-4206-49cf-8342-213c251d0504\") "
Oct 31 01:11:28.308063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.307936    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca" pod "2a53a862-9a21-4196-9011-270d98389276" (UID: "2a53a862-9a21-4196-9011-270d98389276"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:28.308348 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.308322    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca" pod "992b3425-4206-49cf-8342-213c251d0504" (UID: "992b3425-4206-49cf-8342-213c251d0504"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:28.308479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.308454    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2a53a862-9a21-4196-9011-270d98389276-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:28.308479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.308474    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/992b3425-4206-49cf-8342-213c251d0504-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:28.310057 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.310032    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca" pod "992b3425-4206-49cf-8342-213c251d0504" (UID: "992b3425-4206-49cf-8342-213c251d0504"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:28.310130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.310071    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b" pod "2a53a862-9a21-4196-9011-270d98389276" (UID: "2a53a862-9a21-4196-9011-270d98389276"). InnerVolumeSpecName "kube-api-access-hhf7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:28.310506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.310486    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca" pod "2a53a862-9a21-4196-9011-270d98389276" (UID: "2a53a862-9a21-4196-9011-270d98389276"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:28.310568 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.310487    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7" pod "992b3425-4206-49cf-8342-213c251d0504" (UID: "992b3425-4206-49cf-8342-213c251d0504"). InnerVolumeSpecName "kube-api-access-pqkr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:28.409203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.409171    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/992b3425-4206-49cf-8342-213c251d0504-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:28.409203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.409196    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhf7b\" (UniqueName: \"kubernetes.io/projected/2a53a862-9a21-4196-9011-270d98389276-kube-api-access-hhf7b\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:28.409203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.409206    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2a53a862-9a21-4196-9011-270d98389276-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:28.409402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.409215    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqkr7\" (UniqueName: \"kubernetes.io/projected/992b3425-4206-49cf-8342-213c251d0504-kube-api-access-pqkr7\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:28.942418 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.942383    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"
Oct 31 01:11:28.942861 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.942408    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4" event={"ID":"992b3425-4206-49cf-8342-213c251d0504","Type":"ContainerDied","Data":"4016f32b775caff9b1d6cf2c1439fc7ee4f3fdb82509a38b58f0837b623e5059"}
Oct 31 01:11:28.942861 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.942497    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4016f32b775caff9b1d6cf2c1439fc7ee4f3fdb82509a38b58f0837b623e5059"
Oct 31 01:11:28.943949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.943930    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"
Oct 31 01:11:28.944076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.943970    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7" event={"ID":"2a53a862-9a21-4196-9011-270d98389276","Type":"ContainerDied","Data":"ba68bf764fdc76252310ce6b65e3a3a08c95cd95cc033e57b40f632b2400f4c7"}
Oct 31 01:11:28.944076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:28.944001    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba68bf764fdc76252310ce6b65e3a3a08c95cd95cc033e57b40f632b2400f4c7"
Oct 31 01:11:29.077324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.077301    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:29.218498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.218422    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca\") pod \"f5a18cee-2322-4250-8f46-84225409131b\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") "
Oct 31 01:11:29.218677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.218551    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca\") pod \"f5a18cee-2322-4250-8f46-84225409131b\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") "
Oct 31 01:11:29.218677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.218574    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699\" (UniqueName: \"kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699\") pod \"f5a18cee-2322-4250-8f46-84225409131b\" (UID: \"f5a18cee-2322-4250-8f46-84225409131b\") "
Oct 31 01:11:29.218800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.218778    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca" pod "f5a18cee-2322-4250-8f46-84225409131b" (UID: "f5a18cee-2322-4250-8f46-84225409131b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:29.218977 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.218960    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5a18cee-2322-4250-8f46-84225409131b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:29.220742 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.220719    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca" pod "f5a18cee-2322-4250-8f46-84225409131b" (UID: "f5a18cee-2322-4250-8f46-84225409131b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:29.220815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.220785    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699" pod "f5a18cee-2322-4250-8f46-84225409131b" (UID: "f5a18cee-2322-4250-8f46-84225409131b"). InnerVolumeSpecName "kube-api-access-gz699". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:29.319985 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.319952    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5a18cee-2322-4250-8f46-84225409131b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:29.319985 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.319979    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz699\" (UniqueName: \"kubernetes.io/projected/f5a18cee-2322-4250-8f46-84225409131b-kube-api-access-gz699\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:29.952494 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.952471    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"
Oct 31 01:11:29.952906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.952466    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s" event={"ID":"f5a18cee-2322-4250-8f46-84225409131b","Type":"ContainerDied","Data":"5c9aa5818316239ae9ed0eaadfb4af8aec93f07cde24a15ffb97237fb7b98b04"}
Oct 31 01:11:29.952906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:29.952622    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c9aa5818316239ae9ed0eaadfb4af8aec93f07cde24a15ffb97237fb7b98b04"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.660675    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"]
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661535    8029 cpu_manager.go:414] "containerMap: removing container" podUID="f5a18cee-2322-4250-8f46-84225409131b" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661554    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5a18cee-2322-4250-8f46-84225409131b" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661618    8029 cpu_manager.go:414] "containerMap: removing container" podUID="992b3425-4206-49cf-8342-213c251d0504" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661627    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="992b3425-4206-49cf-8342-213c251d0504" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661638    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2a53a862-9a21-4196-9011-270d98389276" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661646    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a53a862-9a21-4196-9011-270d98389276" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661784    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5a18cee-2322-4250-8f46-84225409131b" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661801    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="992b3425-4206-49cf-8342-213c251d0504" containerName="revoke-keys"
Oct 31 01:11:33.664435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.661811    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a53a862-9a21-4196-9011-270d98389276" containerName="revoke-keys"
Oct 31 01:11:33.665944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.665921    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"]
Oct 31 01:11:33.666052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.666025    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.675443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.672689    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:11:33.675443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.672811    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:11:33.675443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.672863    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:11:33.675443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.673341    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:11:33.767175 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.767148    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.767318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.767212    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk5z8\" (UniqueName: \"kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.767318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.767277    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.868223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.868186    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk5z8\" (UniqueName: \"kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.868682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.868659    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.868855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.868802    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.869509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.869488    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.871293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.871272    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.908625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.906490    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk5z8\" (UniqueName: \"kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8\") pod \"maas-api-revoke-sub-75d2b1a8-72jqt\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:33.939671 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.921017    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"]
Oct 31 01:11:33.939671 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.936047    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:33.999661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:33.998667    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"]
Oct 31 01:11:34.065628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.065235    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:34.073278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.073012    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.073278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.073075    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w7bj\" (UniqueName: \"kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.073278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.073121    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.174336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.174287    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.174595 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.174577    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w7bj\" (UniqueName: \"kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.175067 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.175045    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.175695 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.175542    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.179940 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.179800    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.185860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.185813    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w7bj\" (UniqueName: \"kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj\") pod \"maas-api-revoke-sub-d5cb495a-p8nhk\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.365115 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.365074    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:34.465724 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.465363    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"]
Oct 31 01:11:34.661569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.661545    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"]
Oct 31 01:11:34.679907 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:34.679874    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d94cd1_e495_488a_89ae_9a58c4d73762.slice/crio-04a1fd37ab81f5f3f0936829a5e09b7f7cf04c523971739ca239fe6848794408 WatchSource:0}: Error finding container 04a1fd37ab81f5f3f0936829a5e09b7f7cf04c523971739ca239fe6848794408: Status 404 returned error can't find the container with id 04a1fd37ab81f5f3f0936829a5e09b7f7cf04c523971739ca239fe6848794408
Oct 31 01:11:34.763287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.763208    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"]
Oct 31 01:11:34.768734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.768714    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.781270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.781244    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"]
Oct 31 01:11:34.806156 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.806125    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5284\" (UniqueName: \"kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.806376 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.806289    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.806446 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.806415    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.907897 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.907872    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5284\" (UniqueName: \"kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.907998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.907940    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.907998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.907982    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.909007 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.908985    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.910367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.910351    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.915775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.915751    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5284\" (UniqueName: \"kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284\") pod \"maas-api-revoke-sub-f91ed2b0-2zs8r\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:34.928479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.928452    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"]
Oct 31 01:11:34.934002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.933985    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:34.938707 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:34.938684    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"]
Oct 31 01:11:35.009135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.009110    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.009135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.009141    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k79fl\" (UniqueName: \"kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.009388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.009365    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.074379 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.074256    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk" event={"ID":"26d94cd1-e495-488a-89ae-9a58c4d73762","Type":"ContainerStarted","Data":"04a1fd37ab81f5f3f0936829a5e09b7f7cf04c523971739ca239fe6848794408"}
Oct 31 01:11:35.076400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.076367    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt" event={"ID":"a0049f5b-acd7-47d7-a4f8-6e3967b89708","Type":"ContainerStarted","Data":"3dffaa0968a6823ee97b8112a9ed74f38b8dc646f576b9d2f8be74728b054b6b"}
Oct 31 01:11:35.076523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.076430    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt" event={"ID":"a0049f5b-acd7-47d7-a4f8-6e3967b89708","Type":"ContainerStarted","Data":"08dfb25a20f3ae02e8b4751a8ee641e80b459f1b95b5c7db8f8d220b783ab377"}
Oct 31 01:11:35.084447 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.084424    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:35.092630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.092557    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt" podStartSLOduration=2.092545008 podStartE2EDuration="2.092545008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:35.091415312 +0000 UTC m=+892.078527301" watchObservedRunningTime="2026-10-31 01:11:35.092545008 +0000 UTC m=+892.079657046"
Oct 31 01:11:35.111064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.111032    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.111193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.111079    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k79fl\" (UniqueName: \"kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.111335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.111318    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.111794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.111770    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.113587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.113565    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.120321 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.120298    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k79fl\" (UniqueName: \"kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl\") pod \"maas-api-revoke-sub-0dcc5e16-6vjhk\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.250982 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.250953    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:35.375247 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.375218    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"]
Oct 31 01:11:35.379737 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:35.379714    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c72b57f_dc1d_477f_8db7_3866843652ae.slice/crio-e74cb8d8317385b1c62709b6faf9d01731a3fdc20ca106f38d857a2844a343bf WatchSource:0}: Error finding container e74cb8d8317385b1c62709b6faf9d01731a3fdc20ca106f38d857a2844a343bf: Status 404 returned error can't find the container with id e74cb8d8317385b1c62709b6faf9d01731a3fdc20ca106f38d857a2844a343bf
Oct 31 01:11:35.548696 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:35.548670    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"]
Oct 31 01:11:35.549793 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:35.549772    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7be827_646d_439a_92cf_dafb2e988e0b.slice/crio-08c34a456b543994200bb3aca56cebdaff6c57d629c1e0a4ac5b58ccc050bf93 WatchSource:0}: Error finding container 08c34a456b543994200bb3aca56cebdaff6c57d629c1e0a4ac5b58ccc050bf93: Status 404 returned error can't find the container with id 08c34a456b543994200bb3aca56cebdaff6c57d629c1e0a4ac5b58ccc050bf93
Oct 31 01:11:36.086520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.086433    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r" event={"ID":"2c72b57f-dc1d-477f-8db7-3866843652ae","Type":"ContainerStarted","Data":"17d934f9c41dbe8aefc549ce4cf3b7638d47043ee4458062c25b5c322b677224"}
Oct 31 01:11:36.086520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.086491    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r" event={"ID":"2c72b57f-dc1d-477f-8db7-3866843652ae","Type":"ContainerStarted","Data":"e74cb8d8317385b1c62709b6faf9d01731a3fdc20ca106f38d857a2844a343bf"}
Oct 31 01:11:36.088105 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.088075    8029 generic.go:356] "Generic (PLEG): container finished" podID="26d94cd1-e495-488a-89ae-9a58c4d73762" containerID="43b365ae883651edebd8a99bdb7f0c5d8162afad2b5869060025c6b4597870cf" exitCode=0
Oct 31 01:11:36.088225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.088166    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk" event={"ID":"26d94cd1-e495-488a-89ae-9a58c4d73762","Type":"ContainerDied","Data":"43b365ae883651edebd8a99bdb7f0c5d8162afad2b5869060025c6b4597870cf"}
Oct 31 01:11:36.089822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.089800    8029 generic.go:356] "Generic (PLEG): container finished" podID="a0049f5b-acd7-47d7-a4f8-6e3967b89708" containerID="3dffaa0968a6823ee97b8112a9ed74f38b8dc646f576b9d2f8be74728b054b6b" exitCode=0
Oct 31 01:11:36.089908 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.089875    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt" event={"ID":"a0049f5b-acd7-47d7-a4f8-6e3967b89708","Type":"ContainerDied","Data":"3dffaa0968a6823ee97b8112a9ed74f38b8dc646f576b9d2f8be74728b054b6b"}
Oct 31 01:11:36.091516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.091493    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk" event={"ID":"ae7be827-646d-439a-92cf-dafb2e988e0b","Type":"ContainerStarted","Data":"b9b466e436f4e374ecea292f4ac7e96a5ccd32645bdf1dabbd8e4fe36e73d20e"}
Oct 31 01:11:36.091626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.091538    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk" event={"ID":"ae7be827-646d-439a-92cf-dafb2e988e0b","Type":"ContainerStarted","Data":"08c34a456b543994200bb3aca56cebdaff6c57d629c1e0a4ac5b58ccc050bf93"}
Oct 31 01:11:36.110240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.110204    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r" podStartSLOduration=2.110192245 podStartE2EDuration="2.110192245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:36.10879933 +0000 UTC m=+893.095911307" watchObservedRunningTime="2026-10-31 01:11:36.110192245 +0000 UTC m=+893.097304270"
Oct 31 01:11:36.138309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.138270    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk" podStartSLOduration=2.138258553 podStartE2EDuration="2.138258553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:36.1381059 +0000 UTC m=+893.125217881" watchObservedRunningTime="2026-10-31 01:11:36.138258553 +0000 UTC m=+893.125370529"
Oct 31 01:11:36.845358 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.844766    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"]
Oct 31 01:11:36.861825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.852653    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.889041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.888948    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"]
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941347    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941416    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941461    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941539    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941591    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941658    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941735    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.941974    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:36.944372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:36.942013    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzctk\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043177 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043143    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043228    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043320    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043345    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzctk\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043403    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043445    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043489    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043570    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043643    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043869 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043817    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.043869 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.043843    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.044143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.044112    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.044309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.044282    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.044555 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.044395    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.045718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.045693    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.046240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.046219    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.053103 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.053016    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.053202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.053151    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzctk\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.072264 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.072032    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"]
Oct 31 01:11:37.099487 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.093248    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.104774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.103845    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"]
Oct 31 01:11:37.111074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.111044    8029 generic.go:356] "Generic (PLEG): container finished" podID="ae7be827-646d-439a-92cf-dafb2e988e0b" containerID="b9b466e436f4e374ecea292f4ac7e96a5ccd32645bdf1dabbd8e4fe36e73d20e" exitCode=0
Oct 31 01:11:37.111183 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.111170    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk" event={"ID":"ae7be827-646d-439a-92cf-dafb2e988e0b","Type":"ContainerDied","Data":"b9b466e436f4e374ecea292f4ac7e96a5ccd32645bdf1dabbd8e4fe36e73d20e"}
Oct 31 01:11:37.115794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.113327    8029 generic.go:356] "Generic (PLEG): container finished" podID="2c72b57f-dc1d-477f-8db7-3866843652ae" containerID="17d934f9c41dbe8aefc549ce4cf3b7638d47043ee4458062c25b5c322b677224" exitCode=0
Oct 31 01:11:37.115794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.113565    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r" event={"ID":"2c72b57f-dc1d-477f-8db7-3866843652ae","Type":"ContainerDied","Data":"17d934f9c41dbe8aefc549ce4cf3b7638d47043ee4458062c25b5c322b677224"}
Oct 31 01:11:37.144790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.144755    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145010 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.144988    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145142 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.145124    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.145298    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.145444    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.145620    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.145737    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzgxp\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.145890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.145875    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.146019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.146004    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.198628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.198188    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249370    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249516    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249547    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzgxp\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249625    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249655    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249786    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249812    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249833    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.249893    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.251698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.250332    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.252460 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.252436    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.253214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.253189    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.254148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.253863    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.254148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.253873    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.254148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.254110    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.255914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.255799    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.263433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.263390    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzgxp\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.267736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.267714    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.313750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.313128    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"]
Oct 31 01:11:37.334793 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.333262    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.339491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.339470    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"]
Oct 31 01:11:37.353323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.348225    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:37.353323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.351462    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s99wl\" (UniqueName: \"kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.353323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.351569    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.353323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.351689    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.395788 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.395311    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:37.432440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.432413    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:37.452651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.452179    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca\") pod \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") "
Oct 31 01:11:37.452651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.452288    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8\" (UniqueName: \"kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8\") pod \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") "
Oct 31 01:11:37.452651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.452322    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj\" (UniqueName: \"kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj\") pod \"26d94cd1-e495-488a-89ae-9a58c4d73762\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") "
Oct 31 01:11:37.452651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.452558    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca\") pod \"26d94cd1-e495-488a-89ae-9a58c4d73762\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") "
Oct 31 01:11:37.452651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.452620    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca\") pod \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\" (UID: \"a0049f5b-acd7-47d7-a4f8-6e3967b89708\") "
Oct 31 01:11:37.453021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.452686    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca\") pod \"26d94cd1-e495-488a-89ae-9a58c4d73762\" (UID: \"26d94cd1-e495-488a-89ae-9a58c4d73762\") "
Oct 31 01:11:37.453188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.453103    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s99wl\" (UniqueName: \"kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.453248 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.453226    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.453549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.453345    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.454040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.454002    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca" pod "a0049f5b-acd7-47d7-a4f8-6e3967b89708" (UID: "a0049f5b-acd7-47d7-a4f8-6e3967b89708"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:37.454233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.454210    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.454359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.454342    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca" pod "26d94cd1-e495-488a-89ae-9a58c4d73762" (UID: "26d94cd1-e495-488a-89ae-9a58c4d73762"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:37.456397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.456372    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj" pod "26d94cd1-e495-488a-89ae-9a58c4d73762" (UID: "26d94cd1-e495-488a-89ae-9a58c4d73762"). InnerVolumeSpecName "kube-api-access-7w7bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:37.456579 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.456546    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca" pod "a0049f5b-acd7-47d7-a4f8-6e3967b89708" (UID: "a0049f5b-acd7-47d7-a4f8-6e3967b89708"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:37.456794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.456754    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8" pod "a0049f5b-acd7-47d7-a4f8-6e3967b89708" (UID: "a0049f5b-acd7-47d7-a4f8-6e3967b89708"). InnerVolumeSpecName "kube-api-access-nk5z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:37.457019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.457000    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca" pod "26d94cd1-e495-488a-89ae-9a58c4d73762" (UID: "26d94cd1-e495-488a-89ae-9a58c4d73762"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:37.459221 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.459189    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.478926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.478870    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s99wl\" (UniqueName: \"kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl\") pod \"maas-api-revoke-sub-3d47f5ee-kzv4h\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.554645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.554587    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:37.554645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.554646    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:37.554826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.554656    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26d94cd1-e495-488a-89ae-9a58c4d73762-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:37.554826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.554666    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0049f5b-acd7-47d7-a4f8-6e3967b89708-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:37.554826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.554674    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk5z8\" (UniqueName: \"kubernetes.io/projected/a0049f5b-acd7-47d7-a4f8-6e3967b89708-kube-api-access-nk5z8\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:37.554826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.554685    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w7bj\" (UniqueName: \"kubernetes.io/projected/26d94cd1-e495-488a-89ae-9a58c4d73762-kube-api-access-7w7bj\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:37.612167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.612132    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"]
Oct 31 01:11:37.613480 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.613447    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:37.613558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.613512    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:37.613558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.613538    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:37.664668 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.664642    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:37.750840 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:37.750785    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23822fb_a22c_4abe_8fb3_abe2879cdef9.slice/crio-74541639db952a09e033d5dd3f4b094b7ccbb47d2ab95760d12d4443db9a3658 WatchSource:0}: Error finding container 74541639db952a09e033d5dd3f4b094b7ccbb47d2ab95760d12d4443db9a3658: Status 404 returned error can't find the container with id 74541639db952a09e033d5dd3f4b094b7ccbb47d2ab95760d12d4443db9a3658
Oct 31 01:11:37.762025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.756992    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:37.762025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.757077    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:37.762025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.757119    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:37.766898 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.765530    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"]
Oct 31 01:11:37.976506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:37.976484    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"]
Oct 31 01:11:38.129692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.128541    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" event={"ID":"3a7b580d-091c-4735-aedf-5e8964bbd863","Type":"ContainerStarted","Data":"a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae"}
Oct 31 01:11:38.129692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.128627    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" event={"ID":"3a7b580d-091c-4735-aedf-5e8964bbd863","Type":"ContainerStarted","Data":"c158cfbcd589b738a9d530989db922269df5730afe6adc269b5c0565be6ae081"}
Oct 31 01:11:38.141054 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.140728    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"
Oct 31 01:11:38.141242 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.140759    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk" event={"ID":"26d94cd1-e495-488a-89ae-9a58c4d73762","Type":"ContainerDied","Data":"04a1fd37ab81f5f3f0936829a5e09b7f7cf04c523971739ca239fe6848794408"}
Oct 31 01:11:38.141426 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.141410    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04a1fd37ab81f5f3f0936829a5e09b7f7cf04c523971739ca239fe6848794408"
Oct 31 01:11:38.147140 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.147061    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h" event={"ID":"828145cb-f190-4925-b4a9-c723ea6dcb60","Type":"ContainerStarted","Data":"1d0654d33a2b2082a55035ec9f1ea802c3787559596ff6c2affb3defdbeef395"}
Oct 31 01:11:38.150637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.149178    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt" event={"ID":"a0049f5b-acd7-47d7-a4f8-6e3967b89708","Type":"ContainerDied","Data":"08dfb25a20f3ae02e8b4751a8ee641e80b459f1b95b5c7db8f8d220b783ab377"}
Oct 31 01:11:38.150637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.149222    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08dfb25a20f3ae02e8b4751a8ee641e80b459f1b95b5c7db8f8d220b783ab377"
Oct 31 01:11:38.150637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.149304    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"
Oct 31 01:11:38.156370 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.156336    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" event={"ID":"d23822fb-a22c-4abe-8fb3-abe2879cdef9","Type":"ContainerStarted","Data":"ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa"}
Oct 31 01:11:38.156523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.156506    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" event={"ID":"d23822fb-a22c-4abe-8fb3-abe2879cdef9","Type":"ContainerStarted","Data":"74541639db952a09e033d5dd3f4b094b7ccbb47d2ab95760d12d4443db9a3658"}
Oct 31 01:11:38.163538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.162722    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podStartSLOduration=2.162584749 podStartE2EDuration="2.162584749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:38.157264536 +0000 UTC m=+895.144376523" watchObservedRunningTime="2026-10-31 01:11:38.162584749 +0000 UTC m=+895.149696725"
Oct 31 01:11:38.199881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.199825    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podStartSLOduration=1.1998097 podStartE2EDuration="1.1998097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:38.195075806 +0000 UTC m=+895.182187783" watchObservedRunningTime="2026-10-31 01:11:38.1998097 +0000 UTC m=+895.186921676"
Oct 31 01:11:38.201492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.200517    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:38.201739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.201710    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:38.201800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.201769    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:38.460638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.460532    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:38.470649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.469585    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:38.470649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.469671    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:38.475886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.475858    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:38.540666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.540640    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:38.633651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.627005    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca\") pod \"ae7be827-646d-439a-92cf-dafb2e988e0b\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") "
Oct 31 01:11:38.633651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.627095    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl\" (UniqueName: \"kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl\") pod \"ae7be827-646d-439a-92cf-dafb2e988e0b\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") "
Oct 31 01:11:38.633651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.627165    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284\" (UniqueName: \"kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284\") pod \"2c72b57f-dc1d-477f-8db7-3866843652ae\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") "
Oct 31 01:11:38.633651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.627209    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca\") pod \"ae7be827-646d-439a-92cf-dafb2e988e0b\" (UID: \"ae7be827-646d-439a-92cf-dafb2e988e0b\") "
Oct 31 01:11:38.633651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.627245    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca\") pod \"2c72b57f-dc1d-477f-8db7-3866843652ae\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") "
Oct 31 01:11:38.633651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.627425    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca\") pod \"2c72b57f-dc1d-477f-8db7-3866843652ae\" (UID: \"2c72b57f-dc1d-477f-8db7-3866843652ae\") "
Oct 31 01:11:38.635468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.635283    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca" pod "ae7be827-646d-439a-92cf-dafb2e988e0b" (UID: "ae7be827-646d-439a-92cf-dafb2e988e0b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:38.635707 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.635644    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca" pod "2c72b57f-dc1d-477f-8db7-3866843652ae" (UID: "2c72b57f-dc1d-477f-8db7-3866843652ae"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:38.637018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.635724    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca" pod "2c72b57f-dc1d-477f-8db7-3866843652ae" (UID: "2c72b57f-dc1d-477f-8db7-3866843652ae"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:38.638120 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.638081    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca" pod "ae7be827-646d-439a-92cf-dafb2e988e0b" (UID: "ae7be827-646d-439a-92cf-dafb2e988e0b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:38.639275 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.639254    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl" pod "ae7be827-646d-439a-92cf-dafb2e988e0b" (UID: "ae7be827-646d-439a-92cf-dafb2e988e0b"). InnerVolumeSpecName "kube-api-access-k79fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:38.643634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.642806    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284" pod "2c72b57f-dc1d-477f-8db7-3866843652ae" (UID: "2c72b57f-dc1d-477f-8db7-3866843652ae"). InnerVolumeSpecName "kube-api-access-d5284". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:38.731411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.730581    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:38.731411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.731262    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:38.731411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.731284    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k79fl\" (UniqueName: \"kubernetes.io/projected/ae7be827-646d-439a-92cf-dafb2e988e0b-kube-api-access-k79fl\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:38.731411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.731299    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5284\" (UniqueName: \"kubernetes.io/projected/2c72b57f-dc1d-477f-8db7-3866843652ae-kube-api-access-d5284\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:38.731411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.731317    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae7be827-646d-439a-92cf-dafb2e988e0b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:38.731411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:38.731331    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c72b57f-dc1d-477f-8db7-3866843652ae-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:39.163877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.163842    8029 generic.go:356] "Generic (PLEG): container finished" podID="828145cb-f190-4925-b4a9-c723ea6dcb60" containerID="5f68ee52d1da5db9879090e9d409bfd439104c70b8a49308ac4e776e15c06bff" exitCode=0
Oct 31 01:11:39.164297 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.163930    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h" event={"ID":"828145cb-f190-4925-b4a9-c723ea6dcb60","Type":"ContainerDied","Data":"5f68ee52d1da5db9879090e9d409bfd439104c70b8a49308ac4e776e15c06bff"}
Oct 31 01:11:39.166025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.165971    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"
Oct 31 01:11:39.166025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.166006    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk" event={"ID":"ae7be827-646d-439a-92cf-dafb2e988e0b","Type":"ContainerDied","Data":"08c34a456b543994200bb3aca56cebdaff6c57d629c1e0a4ac5b58ccc050bf93"}
Oct 31 01:11:39.166217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.166047    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08c34a456b543994200bb3aca56cebdaff6c57d629c1e0a4ac5b58ccc050bf93"
Oct 31 01:11:39.172752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.167623    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r" event={"ID":"2c72b57f-dc1d-477f-8db7-3866843652ae","Type":"ContainerDied","Data":"e74cb8d8317385b1c62709b6faf9d01731a3fdc20ca106f38d857a2844a343bf"}
Oct 31 01:11:39.172752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.167647    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"
Oct 31 01:11:39.172752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.167659    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e74cb8d8317385b1c62709b6faf9d01731a3fdc20ca106f38d857a2844a343bf"
Oct 31 01:11:39.218845 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.207190    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:39.218845 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.207238    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:39.433294 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.433258    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:39.433484 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:39.433312    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:40.216629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.202111    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:40.216629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.202173    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:40.435165 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.435066    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:40.435165 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.435129    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:40.543993 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.543951    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:40.653969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.653936    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca\") pod \"828145cb-f190-4925-b4a9-c723ea6dcb60\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") "
Oct 31 01:11:40.654152 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.654034    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca\") pod \"828145cb-f190-4925-b4a9-c723ea6dcb60\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") "
Oct 31 01:11:40.654253 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.654233    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl\" (UniqueName: \"kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl\") pod \"828145cb-f190-4925-b4a9-c723ea6dcb60\" (UID: \"828145cb-f190-4925-b4a9-c723ea6dcb60\") "
Oct 31 01:11:40.654397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.654370    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca" pod "828145cb-f190-4925-b4a9-c723ea6dcb60" (UID: "828145cb-f190-4925-b4a9-c723ea6dcb60"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:40.654672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.654651    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:40.656302 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.656270    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca" pod "828145cb-f190-4925-b4a9-c723ea6dcb60" (UID: "828145cb-f190-4925-b4a9-c723ea6dcb60"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:40.658001 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.657973    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl" pod "828145cb-f190-4925-b4a9-c723ea6dcb60" (UID: "828145cb-f190-4925-b4a9-c723ea6dcb60"). InnerVolumeSpecName "kube-api-access-s99wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:40.756155 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.756073    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s99wl\" (UniqueName: \"kubernetes.io/projected/828145cb-f190-4925-b4a9-c723ea6dcb60-kube-api-access-s99wl\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:40.756155 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:40.756100    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/828145cb-f190-4925-b4a9-c723ea6dcb60-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:41.201915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.200343    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:41.201915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.200407    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:41.230797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.230771    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"
Oct 31 01:11:41.232595 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.232571    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h" event={"ID":"828145cb-f190-4925-b4a9-c723ea6dcb60","Type":"ContainerDied","Data":"1d0654d33a2b2082a55035ec9f1ea802c3787559596ff6c2affb3defdbeef395"}
Oct 31 01:11:41.232799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.232783    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d0654d33a2b2082a55035ec9f1ea802c3787559596ff6c2affb3defdbeef395"
Oct 31 01:11:41.433528 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.433487    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:41.433732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:41.433543    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:42.223627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:42.211986    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:42.223627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:42.212057    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:42.433834 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:42.433795    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:42.434221 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:42.433857    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:43.232513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:43.232419    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:43.232733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:43.232509    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:43.343065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:43.343037    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:11:43.352611 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:43.352574    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:11:43.433136 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:43.433088    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:43.433323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:43.433160    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:44.199433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:44.199392    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 01:11:44.199848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:44.199449    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 01:11:44.433857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:44.433808    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 31 01:11:44.434060 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:44.433876    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 31 01:11:45.203752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:45.203723    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:45.204169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:45.203999    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:45.204722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:45.204705    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:11:45.437103 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:45.437075    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:45.437372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:45.437344    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:45.438107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:45.438087    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:11:49.443324 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:11:49.443244    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:11:49.991340 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991308    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"]
Oct 31 01:11:49.991883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991869    8029 cpu_manager.go:414] "containerMap: removing container" podUID="828145cb-f190-4925-b4a9-c723ea6dcb60" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991885    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="828145cb-f190-4925-b4a9-c723ea6dcb60" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991898    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2c72b57f-dc1d-477f-8db7-3866843652ae" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991903    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c72b57f-dc1d-477f-8db7-3866843652ae" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991916    8029 cpu_manager.go:414] "containerMap: removing container" podUID="ae7be827-646d-439a-92cf-dafb2e988e0b" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991921    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae7be827-646d-439a-92cf-dafb2e988e0b" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991931    8029 cpu_manager.go:414] "containerMap: removing container" podUID="26d94cd1-e495-488a-89ae-9a58c4d73762" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991936    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26d94cd1-e495-488a-89ae-9a58c4d73762" containerName="revoke-keys"
Oct 31 01:11:49.991956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991958    8029 cpu_manager.go:414] "containerMap: removing container" podUID="a0049f5b-acd7-47d7-a4f8-6e3967b89708" containerName="revoke-keys"
Oct 31 01:11:49.992218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.991964    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0049f5b-acd7-47d7-a4f8-6e3967b89708" containerName="revoke-keys"
Oct 31 01:11:49.992218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.992049    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="828145cb-f190-4925-b4a9-c723ea6dcb60" containerName="revoke-keys"
Oct 31 01:11:49.992218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.992057    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c72b57f-dc1d-477f-8db7-3866843652ae" containerName="revoke-keys"
Oct 31 01:11:49.992218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.992064    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae7be827-646d-439a-92cf-dafb2e988e0b" containerName="revoke-keys"
Oct 31 01:11:49.992218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.992069    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="26d94cd1-e495-488a-89ae-9a58c4d73762" containerName="revoke-keys"
Oct 31 01:11:49.992218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.992079    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0049f5b-acd7-47d7-a4f8-6e3967b89708" containerName="revoke-keys"
Oct 31 01:11:49.997021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:49.997005    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.002677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.002542    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:11:50.002677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.002569    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:11:50.002677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.002628    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:11:50.002677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.002655    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:11:50.003074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.003056    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"]
Oct 31 01:11:50.088686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.088651    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.088854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.088701    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t4fr\" (UniqueName: \"kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.088854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.088834    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.189848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.189818    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.190014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.189869    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t4fr\" (UniqueName: \"kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.190014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.189951    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.190662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.190591    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.192473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.192443    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.197551 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.197530    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t4fr\" (UniqueName: \"kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr\") pod \"maas-api-revoke-sub-7487971e-dvvbq\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.313415 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.313348    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:50.594282 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:50.594260    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"]
Oct 31 01:11:50.595816 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:50.595779    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0218cbb3_128f_47fe_bcc9_a6b1f29e7ec5.slice/crio-eed991e066557298d0c9418d29f1a307c2a365119a189956fbdab1a63ac6a9a5 WatchSource:0}: Error finding container eed991e066557298d0c9418d29f1a307c2a365119a189956fbdab1a63ac6a9a5: Status 404 returned error can't find the container with id eed991e066557298d0c9418d29f1a307c2a365119a189956fbdab1a63ac6a9a5
Oct 31 01:11:51.352623 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.352572    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq" event={"ID":"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5","Type":"ContainerStarted","Data":"9b43d07f05407a34cc93546aa56e8efbba2c578281e5f5a0dbc0f53f2b5eb886"}
Oct 31 01:11:51.352798 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.352662    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq" event={"ID":"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5","Type":"ContainerStarted","Data":"eed991e066557298d0c9418d29f1a307c2a365119a189956fbdab1a63ac6a9a5"}
Oct 31 01:11:51.370085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.370033    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq" podStartSLOduration=2.370016165 podStartE2EDuration="2.370016165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:51.367470185 +0000 UTC m=+908.354582174" watchObservedRunningTime="2026-10-31 01:11:51.370016165 +0000 UTC m=+908.357128146"
Oct 31 01:11:51.605382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.605305    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"]
Oct 31 01:11:51.609892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.609870    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.620742 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.620712    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"]
Oct 31 01:11:51.709818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.709790    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710006 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.709839    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710006 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.709871    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.710029    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.710067    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.710093    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.710152    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.710267    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.710353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.710315    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9cjd\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.755296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.755257    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"]
Oct 31 01:11:51.759433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.759409    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.793102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.793067    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"]
Oct 31 01:11:51.811280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811237    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811283    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811303    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811466    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwq4h\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811518    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811723 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811541    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.811723 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811639    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811723 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811669    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.811878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811742    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9cjd\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.811878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811810    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.811878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811858    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.812068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811938    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.811983    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812038    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812055    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812072    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812095    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812159    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.812449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812337    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812431    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.812653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812631    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.813020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.812994    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.813125 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.813108    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.814094 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.814073    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.814769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.814752    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.826538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.826510    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9cjd\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.828783 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.828757    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:51.913184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913112    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913216    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913326    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913393    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913445    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913469    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913501    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwq4h\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913552    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913627    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913689    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.913752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913725    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.914005 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.913968    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.914130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.914034    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.914130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.914067    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.915934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.915915    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.916068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.916053    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.928939 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.928915    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwq4h\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.929091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.929072    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:51.947684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:51.947657    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:52.073652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.073625    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:52.232630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.232591    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"]
Oct 31 01:11:52.233368 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:52.233341    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87692b05_4276_4081_a63c_60cd3489b10b.slice/crio-9f17aa2c71b30560ec0bf8e0fd0142057792aaa6b7fcca9965d16c471fae7fa7 WatchSource:0}: Error finding container 9f17aa2c71b30560ec0bf8e0fd0142057792aaa6b7fcca9965d16c471fae7fa7: Status 404 returned error can't find the container with id 9f17aa2c71b30560ec0bf8e0fd0142057792aaa6b7fcca9965d16c471fae7fa7
Oct 31 01:11:52.235987 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.235956    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.236102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.236040    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.236102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.236085    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.364490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.364436    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" event={"ID":"87692b05-4276-4081-a63c-60cd3489b10b","Type":"ContainerStarted","Data":"5184b2c36b80dc5962ef2a49129e7bc8de84845e483e141b6335d2bc51931ac1"}
Oct 31 01:11:52.364490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.364498    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" event={"ID":"87692b05-4276-4081-a63c-60cd3489b10b","Type":"ContainerStarted","Data":"9f17aa2c71b30560ec0bf8e0fd0142057792aaa6b7fcca9965d16c471fae7fa7"}
Oct 31 01:11:52.371646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.368301    8029 generic.go:356] "Generic (PLEG): container finished" podID="0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" containerID="9b43d07f05407a34cc93546aa56e8efbba2c578281e5f5a0dbc0f53f2b5eb886" exitCode=0
Oct 31 01:11:52.371646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.368371    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq" event={"ID":"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5","Type":"ContainerDied","Data":"9b43d07f05407a34cc93546aa56e8efbba2c578281e5f5a0dbc0f53f2b5eb886"}
Oct 31 01:11:52.371646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.370774    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"]
Oct 31 01:11:52.372640 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:52.372583    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19c3781_1aad_4392_9924_4a084cde6faf.slice/crio-102617539f49e12b64a7d96c2b1d6764549fc37c666b79d5550d5b12851aee4a WatchSource:0}: Error finding container 102617539f49e12b64a7d96c2b1d6764549fc37c666b79d5550d5b12851aee4a: Status 404 returned error can't find the container with id 102617539f49e12b64a7d96c2b1d6764549fc37c666b79d5550d5b12851aee4a
Oct 31 01:11:52.375636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.375576    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.375702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.375679    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.375745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.375724    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:11:52.389894 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.389835    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podStartSLOduration=1.389816562 podStartE2EDuration="1.389816562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:52.386343091 +0000 UTC m=+909.373455067" watchObservedRunningTime="2026-10-31 01:11:52.389816562 +0000 UTC m=+909.376928539"
Oct 31 01:11:52.960645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:52.958928    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:53.403777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.403728    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" event={"ID":"f19c3781-1aad-4392-9924-4a084cde6faf","Type":"ContainerStarted","Data":"4ad5b72a515e25c3ef76e605fdad322bcb1ac4d52b1568ca137895dc26c0f5d9"}
Oct 31 01:11:53.404101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.404073    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" event={"ID":"f19c3781-1aad-4392-9924-4a084cde6faf","Type":"ContainerStarted","Data":"102617539f49e12b64a7d96c2b1d6764549fc37c666b79d5550d5b12851aee4a"}
Oct 31 01:11:53.432771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.432692    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podStartSLOduration=2.432670017 podStartE2EDuration="2.432670017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:53.431047879 +0000 UTC m=+910.418159857" watchObservedRunningTime="2026-10-31 01:11:53.432670017 +0000 UTC m=+910.419781987"
Oct 31 01:11:53.675755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.675730    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:53.747206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.747174    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca\") pod \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") "
Oct 31 01:11:53.747733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.747695    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr\" (UniqueName: \"kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr\") pod \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") "
Oct 31 01:11:53.747838 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.747792    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca\") pod \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\" (UID: \"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5\") "
Oct 31 01:11:53.748585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.748554    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca" pod "0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" (UID: "0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:11:53.760862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.756322    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca" pod "0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" (UID: "0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:11:53.761009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.760932    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr" pod "0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" (UID: "0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5"). InnerVolumeSpecName "kube-api-access-9t4fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:11:53.851301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.851241    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:53.851301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.851275    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t4fr\" (UniqueName: \"kubernetes.io/projected/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-kube-api-access-9t4fr\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:53.851301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.851290    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:11:53.949821 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.949732    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 01:11:53.949821 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:53.949786    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 01:11:54.074554 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.074526    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:54.076084 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.076057    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 31 01:11:54.076294 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.076266    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 31 01:11:54.442652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.442581    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq" event={"ID":"0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5","Type":"ContainerDied","Data":"eed991e066557298d0c9418d29f1a307c2a365119a189956fbdab1a63ac6a9a5"}
Oct 31 01:11:54.442864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.442774    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eed991e066557298d0c9418d29f1a307c2a365119a189956fbdab1a63ac6a9a5"
Oct 31 01:11:54.442954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.442901    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"
Oct 31 01:11:54.948507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.948476    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 01:11:54.948709 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:54.948525    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 01:11:55.074729 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:55.074697    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 31 01:11:55.075117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:55.074750    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 31 01:11:55.948963 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:55.948913    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 01:11:55.949134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:55.948980    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 01:11:56.074150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:56.074116    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 31 01:11:56.074315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:56.074173    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 31 01:11:56.948733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:56.948679    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 01:11:56.949254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:56.948753    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 01:11:57.074491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:57.074458    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 31 01:11:57.074681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:57.074504    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 31 01:11:57.948251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:57.948216    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 01:11:57.948444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:57.948270    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 01:11:58.031583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.031522    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"]
Oct 31 01:11:58.033644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.033573    8029 cpu_manager.go:414] "containerMap: removing container" podUID="0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" containerName="revoke-keys"
Oct 31 01:11:58.033795 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.033644    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" containerName="revoke-keys"
Oct 31 01:11:58.033922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.033902    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" containerName="revoke-keys"
Oct 31 01:11:58.050416 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.047424    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.052835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.052809    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"]
Oct 31 01:11:58.062166 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.053061    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-acdaed\"/\"distinct-w0-acdaed-kserve-self-signed-certs\""
Oct 31 01:11:58.074179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.074154    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 31 01:11:58.074298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.074200    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 31 01:11:58.155305 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.155271    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.155469 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.155343    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.155469 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.155413    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm94t\" (UniqueName: \"kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.155469 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.155464    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.155572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.155538    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.155619 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.155573    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257279    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257573 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257424    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257573 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257551    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257699    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257725    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257804    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm94t\" (UniqueName: \"kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.257994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.257966    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.258172 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.258076    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.258593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.258568    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.260833 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.260809    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.265168 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.265142    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.265860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.265835    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm94t\" (UniqueName: \"kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t\") pod \"distinct-w0-acdaed-kserve-845c67f5c9-qh9k7\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") " pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.375359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.375324    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:11:58.673128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.673100    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"]
Oct 31 01:11:58.674305 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:11:58.674270    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad74513f_78eb_4047_9421_79740761bbd6.slice/crio-9f5233eda531782288296b3ab439856ca43608e7232685ea03cf939db171ec80 WatchSource:0}: Error finding container 9f5233eda531782288296b3ab439856ca43608e7232685ea03cf939db171ec80: Status 404 returned error can't find the container with id 9f5233eda531782288296b3ab439856ca43608e7232685ea03cf939db171ec80
Oct 31 01:11:58.948021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.947943    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 31 01:11:58.948168 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:58.948021    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 31 01:11:59.344363 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.344288    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:59.344880 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.344861    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:59.345755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.345728    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:11:59.492882 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.492850    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" event={"ID":"ad74513f-78eb-4047-9421-79740761bbd6","Type":"ContainerStarted","Data":"506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585"}
Oct 31 01:11:59.493037 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.492891    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" event={"ID":"ad74513f-78eb-4047-9421-79740761bbd6","Type":"ContainerStarted","Data":"9f5233eda531782288296b3ab439856ca43608e7232685ea03cf939db171ec80"}
Oct 31 01:11:59.519215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.519147    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podStartSLOduration=1.519121449 podStartE2EDuration="1.519121449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:11:59.514576677 +0000 UTC m=+916.501688654" watchObservedRunningTime="2026-10-31 01:11:59.519121449 +0000 UTC m=+916.506233427"
Oct 31 01:11:59.953773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.953747    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:59.954150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.954114    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:11:59.954827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:11:59.954806    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:12:00.301250 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.301177    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"]
Oct 31 01:12:00.305398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.305382    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.309474 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.309409    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:12:00.309956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.309931    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:12:00.311326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.311307    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:12:00.311447 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.311358    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:12:00.315231 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.315088    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"]
Oct 31 01:12:00.376132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.376108    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:12:00.386734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.386712    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.386888 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.386773    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.386936 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.386916    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgh6v\" (UniqueName: \"kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.488659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.488596    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.488820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.488698    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.488820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.488762    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgh6v\" (UniqueName: \"kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.489433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.489413    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.491052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.491029    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.499536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.499512    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgh6v\" (UniqueName: \"kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v\") pod \"maas-api-revoke-sub-4cf2b4d9-jh2db\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.622368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.622334    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:00.904574 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:00.904543    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"]
Oct 31 01:12:00.905650 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:00.905590    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490b1bf5_e143_4bde_9696_35b49dd2b637.slice/crio-51a8823d5abd6f23ae91a09d47ec548b134e2b8fb2f6f50c77580939427616da WatchSource:0}: Error finding container 51a8823d5abd6f23ae91a09d47ec548b134e2b8fb2f6f50c77580939427616da: Status 404 returned error can't find the container with id 51a8823d5abd6f23ae91a09d47ec548b134e2b8fb2f6f50c77580939427616da
Oct 31 01:12:01.492378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.492295    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"]
Oct 31 01:12:01.496893 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.496870    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.509995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.509960    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db" event={"ID":"490b1bf5-e143-4bde-9696-35b49dd2b637","Type":"ContainerStarted","Data":"d628710a306152df69507ebd9a19581db8c9d30791784a86f8fa7f4311f4dba5"}
Oct 31 01:12:01.510564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.510539    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db" event={"ID":"490b1bf5-e143-4bde-9696-35b49dd2b637","Type":"ContainerStarted","Data":"51a8823d5abd6f23ae91a09d47ec548b134e2b8fb2f6f50c77580939427616da"}
Oct 31 01:12:01.514246 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.514215    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7f6d1e\"/\"distinct-w1-7f6d1e-kserve-self-signed-certs\""
Oct 31 01:12:01.516264 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.516241    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"]
Oct 31 01:12:01.603818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.603778    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrfnj\" (UniqueName: \"kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.603978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.603946    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.604079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.604032    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.604161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.604137    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.604219 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.604206    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.604274 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.604228    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.705967 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.705933    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrfnj\" (UniqueName: \"kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706001    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706043    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706097    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706133    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706154    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706454    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706627    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.706751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.706683    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.708831 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.708790    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.709045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.709024    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.716747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.716723    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrfnj\" (UniqueName: \"kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj\") pod \"distinct-w1-7f6d1e-kserve-84988c4445-kp2fw\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") " pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:01.818054 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:01.818030    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:02.099473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.099360    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db" podStartSLOduration=2.099340641 podStartE2EDuration="2.099340641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:01.558916142 +0000 UTC m=+918.546028119" watchObservedRunningTime="2026-10-31 01:12:02.099340641 +0000 UTC m=+919.086452618"
Oct 31 01:12:02.103192 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.103165    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"]
Oct 31 01:12:02.103399 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:02.103376    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod461dfedc_0264_4ba3_8248_b647aa593d60.slice/crio-292a0c43a4e9f39bb278d88258b2b66edd7fd38bc124df262b41826003eb577c WatchSource:0}: Error finding container 292a0c43a4e9f39bb278d88258b2b66edd7fd38bc124df262b41826003eb577c: Status 404 returned error can't find the container with id 292a0c43a4e9f39bb278d88258b2b66edd7fd38bc124df262b41826003eb577c
Oct 31 01:12:02.518928 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.518892    8029 generic.go:356] "Generic (PLEG): container finished" podID="490b1bf5-e143-4bde-9696-35b49dd2b637" containerID="d628710a306152df69507ebd9a19581db8c9d30791784a86f8fa7f4311f4dba5" exitCode=0
Oct 31 01:12:02.519375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.518991    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db" event={"ID":"490b1bf5-e143-4bde-9696-35b49dd2b637","Type":"ContainerDied","Data":"d628710a306152df69507ebd9a19581db8c9d30791784a86f8fa7f4311f4dba5"}
Oct 31 01:12:02.520953 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.520926    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" event={"ID":"461dfedc-0264-4ba3-8248-b647aa593d60","Type":"ContainerStarted","Data":"59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26"}
Oct 31 01:12:02.521076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.520959    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" event={"ID":"461dfedc-0264-4ba3-8248-b647aa593d60","Type":"ContainerStarted","Data":"292a0c43a4e9f39bb278d88258b2b66edd7fd38bc124df262b41826003eb577c"}
Oct 31 01:12:02.554151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.554063    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podStartSLOduration=1.554045116 podStartE2EDuration="1.554045116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:02.553058182 +0000 UTC m=+919.540170159" watchObservedRunningTime="2026-10-31 01:12:02.554045116 +0000 UTC m=+919.541157093"
Oct 31 01:12:02.830684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:02.825729    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:04.649548 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:04.636393    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": context deadline exceeded"
Oct 31 01:12:04.907070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:04.905333    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:05.050659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.049256    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca\") pod \"490b1bf5-e143-4bde-9696-35b49dd2b637\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") "
Oct 31 01:12:05.050659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.049385    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca\") pod \"490b1bf5-e143-4bde-9696-35b49dd2b637\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") "
Oct 31 01:12:05.050659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.049575    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v\" (UniqueName: \"kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v\") pod \"490b1bf5-e143-4bde-9696-35b49dd2b637\" (UID: \"490b1bf5-e143-4bde-9696-35b49dd2b637\") "
Oct 31 01:12:05.062626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.055190    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca" pod "490b1bf5-e143-4bde-9696-35b49dd2b637" (UID: "490b1bf5-e143-4bde-9696-35b49dd2b637"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:12:05.062626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.056191    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:05.071339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.065652    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v" pod "490b1bf5-e143-4bde-9696-35b49dd2b637" (UID: "490b1bf5-e143-4bde-9696-35b49dd2b637"). InnerVolumeSpecName "kube-api-access-kgh6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:12:05.071339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.071174    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca" pod "490b1bf5-e143-4bde-9696-35b49dd2b637" (UID: "490b1bf5-e143-4bde-9696-35b49dd2b637"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:12:05.158235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.158148    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/490b1bf5-e143-4bde-9696-35b49dd2b637-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:05.158235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.158187    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgh6v\" (UniqueName: \"kubernetes.io/projected/490b1bf5-e143-4bde-9696-35b49dd2b637-kube-api-access-kgh6v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:05.753406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.753376    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"
Oct 31 01:12:05.753406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.753393    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db" event={"ID":"490b1bf5-e143-4bde-9696-35b49dd2b637","Type":"ContainerDied","Data":"51a8823d5abd6f23ae91a09d47ec548b134e2b8fb2f6f50c77580939427616da"}
Oct 31 01:12:05.753834 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:05.753439    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51a8823d5abd6f23ae91a09d47ec548b134e2b8fb2f6f50c77580939427616da"
Oct 31 01:12:06.995248 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:06.995219    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"]
Oct 31 01:12:06.996029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:06.996001    8029 cpu_manager.go:414] "containerMap: removing container" podUID="490b1bf5-e143-4bde-9696-35b49dd2b637" containerName="revoke-keys"
Oct 31 01:12:06.996120 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:06.996030    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="490b1bf5-e143-4bde-9696-35b49dd2b637" containerName="revoke-keys"
Oct 31 01:12:06.996234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:06.996220    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="490b1bf5-e143-4bde-9696-35b49dd2b637" containerName="revoke-keys"
Oct 31 01:12:07.000283 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.000263    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.014524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.014498    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"]
Oct 31 01:12:07.034036 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.034006    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"]
Oct 31 01:12:07.038669 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.038648    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.051755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.051726    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"]
Oct 31 01:12:07.081048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081016    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081058    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081079    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081283 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081207    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081290    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081424 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081406    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081500    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081572    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnrp4\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.081651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.081633    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183366 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183336    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183377    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.183556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183412    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183436    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183480    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183528    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.183848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183583    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183663    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.183848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183710    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.183848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183782    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.183848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183837    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183853    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183866    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183957    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.184188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.183989    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnrp4\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184030    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.184188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184138    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184221    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184289    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.184436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184373    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5k5\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.184436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184407    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.184436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184429    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.184666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.184545    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.185849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.185822    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.186181 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.186164    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.192056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.192036    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.192140 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.192056    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnrp4\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.250565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.250501    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"]
Oct 31 01:12:07.256227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.256207    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.283732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.283699    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"]
Oct 31 01:12:07.285846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.285815    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.285973 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.285890    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286057 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286000    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286057 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286031    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286117    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5k5\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286213 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286161    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286213 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286195    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286252    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286302    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286339    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286361    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.286921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286899    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.287003 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286857    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.287041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.286997    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.288638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.288576    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.289031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.289004    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.319372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.319232    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:07.323143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.323108    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.323788 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.323361    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5k5\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.363846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.363817    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:07.388351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388320    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388512 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388369    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388512 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388393    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388539    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388653    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388693    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj9kt\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388738    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388761    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.388875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.388784    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490413    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490648 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490457    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490648 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490473    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490648 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490522    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490658    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490694    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj9kt\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490748    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490779    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.490826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490813    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.491086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.490908    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.491086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.491035    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.491298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.491276    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.491555 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.491531    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.491789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.491761    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.493325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.493303    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.493455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.493436    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.499206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.499183    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj9kt\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.499393 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.499370    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.580222 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.580141    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:07.616735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.616708    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"]
Oct 31 01:12:07.616973 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:07.616952    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda6944e_5abf_4cbb_a38d_1ebc269b6a05.slice/crio-3fc27c137068460ae38a793e933299ad9efb1520c7422408179ba3d1cf81340f WatchSource:0}: Error finding container 3fc27c137068460ae38a793e933299ad9efb1520c7422408179ba3d1cf81340f: Status 404 returned error can't find the container with id 3fc27c137068460ae38a793e933299ad9efb1520c7422408179ba3d1cf81340f
Oct 31 01:12:07.620132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.620103    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.620201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.620167    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.620201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.620194    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.663888 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.663857    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"]
Oct 31 01:12:07.667997 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:07.667955    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67747552_c23c_449f_b7dd_968dbd0e1020.slice/crio-cd4f1a359498d16b8b18881daedba3f1d64afe77cb240f62e36b3aad44b42a6f WatchSource:0}: Error finding container cd4f1a359498d16b8b18881daedba3f1d64afe77cb240f62e36b3aad44b42a6f: Status 404 returned error can't find the container with id cd4f1a359498d16b8b18881daedba3f1d64afe77cb240f62e36b3aad44b42a6f
Oct 31 01:12:07.670725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.670692    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.670807 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.670763    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.670842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.670821    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.771486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.770949    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" event={"ID":"67747552-c23c-449f-b7dd-968dbd0e1020","Type":"ContainerStarted","Data":"fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199"}
Oct 31 01:12:07.771486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.771014    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" event={"ID":"67747552-c23c-449f-b7dd-968dbd0e1020","Type":"ContainerStarted","Data":"cd4f1a359498d16b8b18881daedba3f1d64afe77cb240f62e36b3aad44b42a6f"}
Oct 31 01:12:07.773747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.773720    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" event={"ID":"fda6944e-5abf-4cbb-a38d-1ebc269b6a05","Type":"ContainerStarted","Data":"c1a18ecba2b13d0e00478b5bea8412b8a86c2f98368752b722e50f3961b505bf"}
Oct 31 01:12:07.773865 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.773770    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" event={"ID":"fda6944e-5abf-4cbb-a38d-1ebc269b6a05","Type":"ContainerStarted","Data":"3fc27c137068460ae38a793e933299ad9efb1520c7422408179ba3d1cf81340f"}
Oct 31 01:12:07.791439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.791336    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podStartSLOduration=0.791318345 podStartE2EDuration="791.318345ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:07.788195454 +0000 UTC m=+924.775307432" watchObservedRunningTime="2026-10-31 01:12:07.791318345 +0000 UTC m=+924.778430322"
Oct 31 01:12:07.810658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.810475    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podStartSLOduration=1.810456325 podStartE2EDuration="1.810456325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:07.808949243 +0000 UTC m=+924.796061221" watchObservedRunningTime="2026-10-31 01:12:07.810456325 +0000 UTC m=+924.797568304"
Oct 31 01:12:07.872999 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:07.872969    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f818a5e_0da0_48b8_8a82_4b93068c7232.slice/crio-57864a43b2f45c9eebd1167d47f44724d2ea910709477516a19e31ca2e288043 WatchSource:0}: Error finding container 57864a43b2f45c9eebd1167d47f44724d2ea910709477516a19e31ca2e288043: Status 404 returned error can't find the container with id 57864a43b2f45c9eebd1167d47f44724d2ea910709477516a19e31ca2e288043
Oct 31 01:12:07.874302 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.874269    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"]
Oct 31 01:12:07.876263 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.876226    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.876334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.876313    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:07.876382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:07.876366    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:08.320674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.320583    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:08.321959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.321928    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:08.322067 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.321976    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:08.365024 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.364992    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:08.366480 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.366453    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:08.366612 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.366500    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:08.376316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.376288    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:12:08.400023 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.399984    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:12:08.859617 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.859557    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" event={"ID":"8f818a5e-0da0-48b8-8a82-4b93068c7232","Type":"ContainerStarted","Data":"ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411"}
Oct 31 01:12:08.859856 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.859838    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" event={"ID":"8f818a5e-0da0-48b8-8a82-4b93068c7232","Type":"ContainerStarted","Data":"57864a43b2f45c9eebd1167d47f44724d2ea910709477516a19e31ca2e288043"}
Oct 31 01:12:08.903097 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.900866    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:12:08.917010 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:08.916944    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podStartSLOduration=1.916921267 podStartE2EDuration="1.916921267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:08.891291527 +0000 UTC m=+925.878403554" watchObservedRunningTime="2026-10-31 01:12:08.916921267 +0000 UTC m=+925.904033244"
Oct 31 01:12:09.321844 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.321749    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:09.322424 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.322391    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:09.365059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.365030    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:09.365269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.365250    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:09.588770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.588677    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:09.592730 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.592698    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 01:12:09.592951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:09.592923    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 01:12:10.322059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:10.322019    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:10.322658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:10.322627    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:10.365164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:10.365131    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:10.365336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:10.365181    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:10.583683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:10.580934    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 01:12:10.583683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:10.580993    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 01:12:11.323214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.323180    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:11.323932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.323897    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:11.366080 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.366046    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:11.366378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.366350    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:11.581174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.581076    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 01:12:11.601628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.581455    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 01:12:11.823627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.822732    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:11.869086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:11.869056    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:12.098626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.089948    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:12:12.321082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.320991    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:12.321329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.321299    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:12.364995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.364962    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:12.365488 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.365458    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:12.581128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.581034    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 01:12:12.581403 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:12.581371    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 01:12:13.341629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:13.329036    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:13.341629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:13.329099    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:13.364636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:13.364583    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:13.364819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:13.364657    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:13.600686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:13.586204    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 01:12:13.600686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:13.586265    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 01:12:14.319889 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:14.319851    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 31 01:12:14.320110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:14.319910    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 31 01:12:14.366728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:14.366557    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 31 01:12:14.366728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:14.366647    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 31 01:12:14.591704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:14.588037    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 01:12:14.591704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:14.588097    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 01:12:15.369995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.369962    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:15.370502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.370430    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:15.371435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.371415    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:12:15.400041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.400005    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:15.400546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.400521    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:15.401726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.401702    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:12:15.584967 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.584936    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:15.585485 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.585460    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:15.586238 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:15.586218    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:12:17.241634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.238033    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"]
Oct 31 01:12:17.286656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.286621    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"]
Oct 31 01:12:17.287048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.286974    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.296314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.296294    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 01:12:17.301671 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.296837    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 01:12:17.326070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.307069    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 01:12:17.326070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.307187    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-clrlv\""
Oct 31 01:12:17.387475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.387428    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.387836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.387812    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49hvn\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.388160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.388134    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.388629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.388583    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.388809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.388788    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.388947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.388928    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.389096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.389081    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.495492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.495388    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49hvn\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.495904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.495859    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.496317 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.496294    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.496529 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.496477    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.496696 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.496676    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.496846 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.496832    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.497053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.497033    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.498787 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.498752    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.606052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.605700    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49hvn\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.606052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.605984    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.606732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.606684    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.620204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.620135    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.620778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.620726    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.621828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.621806    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7kbms\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:17.672893 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:17.672855    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:18.501628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:18.486270    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"]
Oct 31 01:12:18.907630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:18.885095    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:12:19.146663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:19.143700    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:12:19.148523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:19.147687    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:12:19.182691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:19.178699    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:12:19.226630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:19.198779    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:12:19.529191 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:12:19.529069    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:12:19.555331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:19.555298    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms" event={"ID":"fa39e042-7d7a-44df-b93a-4a12c30c8708","Type":"ContainerStarted","Data":"0d224ce94694a6b34fa6ad246ef17e7d1986e44db5f2cfc75666430aa88868e7"}
Oct 31 01:12:22.964943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:22.964902    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:22.965293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:22.964973    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:23.588754 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:23.588715    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms" event={"ID":"fa39e042-7d7a-44df-b93a-4a12c30c8708","Type":"ContainerStarted","Data":"8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5"}
Oct 31 01:12:23.588955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:23.588796    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:23.611661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:23.611593    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms" podStartSLOduration=2.257039353 podStartE2EDuration="6.611581076s" totalImagesPullingTime="4.354541723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:23.609187574 +0000 UTC m=+940.596299551" watchObservedRunningTime="2026-10-31 01:12:23.611581076 +0000 UTC m=+940.598693052"
Oct 31 01:12:24.051897 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.051867    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"]
Oct 31 01:12:24.056014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.055990    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.059651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.059595    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:12:24.060066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.060045    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:12:24.060237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.060085    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:12:24.060327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.060167    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:12:24.073775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.068119    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"]
Oct 31 01:12:24.157118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.157082    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6h5f\" (UniqueName: \"kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.157298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.157195    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.157298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.157229    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.258482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.258446    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6h5f\" (UniqueName: \"kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.258698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.258523    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.258698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.258548    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.259422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.259400    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.261383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.261360    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.266559 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.266537    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6h5f\" (UniqueName: \"kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f\") pod \"maas-api-revoke-sub-5eb8a263-9hk29\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.377259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.377225    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:24.602044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.602011    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:12:24.706716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:24.706689    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"]
Oct 31 01:12:25.609377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:25.609329    8029 generic.go:356] "Generic (PLEG): container finished" podID="03f30b7e-ac75-4800-88c2-788a51708025" containerID="a3db475550412fbb2d397c258ea6876682690cca52879b115948b121bdae4a99" exitCode=0
Oct 31 01:12:25.609815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:25.609405    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29" event={"ID":"03f30b7e-ac75-4800-88c2-788a51708025","Type":"ContainerDied","Data":"a3db475550412fbb2d397c258ea6876682690cca52879b115948b121bdae4a99"}
Oct 31 01:12:25.609815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:25.609462    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29" event={"ID":"03f30b7e-ac75-4800-88c2-788a51708025","Type":"ContainerStarted","Data":"1cd6b11c06bef8b64f6b0ee1376e0cab6a44d1e6df4af9cc555d9ddd0a998fd3"}
Oct 31 01:12:26.801524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.801491    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:26.909650 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.909535    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca\") pod \"03f30b7e-ac75-4800-88c2-788a51708025\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") "
Oct 31 01:12:26.909650 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.909573    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca\") pod \"03f30b7e-ac75-4800-88c2-788a51708025\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") "
Oct 31 01:12:26.909850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.909790    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f\" (UniqueName: \"kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f\") pod \"03f30b7e-ac75-4800-88c2-788a51708025\" (UID: \"03f30b7e-ac75-4800-88c2-788a51708025\") "
Oct 31 01:12:26.909955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.909931    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca" pod "03f30b7e-ac75-4800-88c2-788a51708025" (UID: "03f30b7e-ac75-4800-88c2-788a51708025"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:12:26.910393 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.910368    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:26.911973 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.911953    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca" pod "03f30b7e-ac75-4800-88c2-788a51708025" (UID: "03f30b7e-ac75-4800-88c2-788a51708025"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:12:26.912227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:26.912210    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f" pod "03f30b7e-ac75-4800-88c2-788a51708025" (UID: "03f30b7e-ac75-4800-88c2-788a51708025"). InnerVolumeSpecName "kube-api-access-l6h5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:12:27.012093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:27.012059    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03f30b7e-ac75-4800-88c2-788a51708025-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:27.012093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:27.012090    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6h5f\" (UniqueName: \"kubernetes.io/projected/03f30b7e-ac75-4800-88c2-788a51708025-kube-api-access-l6h5f\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:27.628833 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:27.628799    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29" event={"ID":"03f30b7e-ac75-4800-88c2-788a51708025","Type":"ContainerDied","Data":"1cd6b11c06bef8b64f6b0ee1376e0cab6a44d1e6df4af9cc555d9ddd0a998fd3"}
Oct 31 01:12:27.628981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:27.628872    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1cd6b11c06bef8b64f6b0ee1376e0cab6a44d1e6df4af9cc555d9ddd0a998fd3"
Oct 31 01:12:27.628981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:27.628812    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"
Oct 31 01:12:35.580952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.580906    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"]
Oct 31 01:12:35.581560 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.581545    8029 cpu_manager.go:414] "containerMap: removing container" podUID="03f30b7e-ac75-4800-88c2-788a51708025" containerName="revoke-keys"
Oct 31 01:12:35.581637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.581560    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03f30b7e-ac75-4800-88c2-788a51708025" containerName="revoke-keys"
Oct 31 01:12:35.581701 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.581691    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="03f30b7e-ac75-4800-88c2-788a51708025" containerName="revoke-keys"
Oct 31 01:12:35.586380 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.586363    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.590001 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.589974    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:12:35.590578 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.590559    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:12:35.590793 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.590778    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:12:35.590916 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.590904    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:12:35.594799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.594771    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"]
Oct 31 01:12:35.627047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.627010    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5jp6\" (UniqueName: \"kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.627242 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.627224    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.627311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.627251    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.733038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.732996    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.733289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.733051    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.733289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.733159    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5jp6\" (UniqueName: \"kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.733785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.733762    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.735715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.735693    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.747339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.747311    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5jp6\" (UniqueName: \"kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6\") pod \"maas-api-revoke-sub-206b91f3-n2m67\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:35.905488 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:35.905455    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:36.188179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:36.188150    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"]
Oct 31 01:12:36.189565 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:36.189542    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cba2b7_af0f_4a27_889f_cd6b82bc63ad.slice/crio-7de99bad36d2bfe040f774122aa39e96e0ed41b11d5d46b78d5470b76c99a3f7 WatchSource:0}: Error finding container 7de99bad36d2bfe040f774122aa39e96e0ed41b11d5d46b78d5470b76c99a3f7: Status 404 returned error can't find the container with id 7de99bad36d2bfe040f774122aa39e96e0ed41b11d5d46b78d5470b76c99a3f7
Oct 31 01:12:36.696099 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:36.696067    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67" event={"ID":"86cba2b7-af0f-4a27-889f-cd6b82bc63ad","Type":"ContainerStarted","Data":"90fee6d375f87f0f557e0be384f5560bb769c5b1501e79f52b8df14ae3d5616a"}
Oct 31 01:12:36.696515 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:36.696125    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67" event={"ID":"86cba2b7-af0f-4a27-889f-cd6b82bc63ad","Type":"ContainerStarted","Data":"7de99bad36d2bfe040f774122aa39e96e0ed41b11d5d46b78d5470b76c99a3f7"}
Oct 31 01:12:36.714218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:36.714144    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67" podStartSLOduration=1.714125738 podStartE2EDuration="1.714125738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:36.713317666 +0000 UTC m=+953.700429645" watchObservedRunningTime="2026-10-31 01:12:36.714125738 +0000 UTC m=+953.701237714"
Oct 31 01:12:37.204185 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.204129    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"]
Oct 31 01:12:37.234181 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.234146    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"]
Oct 31 01:12:37.235987 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.235965    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.248463 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.248402    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"]
Oct 31 01:12:37.248620 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.248579    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.252327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.252302    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"]
Oct 31 01:12:37.282048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282012    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282209 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282064    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282209 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282166    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282209 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282202    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282281    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282315    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282418    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282443    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282472    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282503    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282528    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282579    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxmcm\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282629    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282728    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkj9v\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282777    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282812    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.282949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282839    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.282949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.282885    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.384397    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.384445    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.384477    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.384507    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.384535    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.384581    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxmcm\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.388226    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.388531    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.388786    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.389037    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.404624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.389285    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405372    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405484    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkj9v\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405533    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405569    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405592    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405660    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405710    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405737    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405847    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405882    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.405979    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.406015    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.411592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.407046    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.412438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.412343    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.412959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.412935    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.413591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.413482    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.414205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.414179    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.452673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.443437    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkj9v\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.452673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.448172    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.456297 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.456180    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.506680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.506444    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.583169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.580242    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.583169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.580518    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.583169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.580741    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.701628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.699355    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxmcm\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm\") pod \"e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") " pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.802687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.785555    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:37.894712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.894166    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:37.916711 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.899717    8029 generic.go:356] "Generic (PLEG): container finished" podID="86cba2b7-af0f-4a27-889f-cd6b82bc63ad" containerID="90fee6d375f87f0f557e0be384f5560bb769c5b1501e79f52b8df14ae3d5616a" exitCode=0
Oct 31 01:12:37.916711 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:37.899864    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67" event={"ID":"86cba2b7-af0f-4a27-889f-cd6b82bc63ad","Type":"ContainerDied","Data":"90fee6d375f87f0f557e0be384f5560bb769c5b1501e79f52b8df14ae3d5616a"}
Oct 31 01:12:38.993545 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:38.971739    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"]
Oct 31 01:12:39.071822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.071785    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:39.072084 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.072065    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:39.072209 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.072195    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:39.429403 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.429367    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"]
Oct 31 01:12:39.437179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.437146    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" event={"ID":"d2813b0a-d570-425a-9523-778af9495993","Type":"ContainerStarted","Data":"3f83f7da5dce113eb90cc0c8941f7b67f2d36397d4f397af72ae77f9b80615d0"}
Oct 31 01:12:39.560991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.555684    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:39.560991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.555769    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:39.560991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.555837    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:39.928724 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.928662    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:39.985630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.983262    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca\") pod \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") "
Oct 31 01:12:39.985630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.983383    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca\") pod \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") "
Oct 31 01:12:39.985630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.983424    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6\" (UniqueName: \"kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6\") pod \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\" (UID: \"86cba2b7-af0f-4a27-889f-cd6b82bc63ad\") "
Oct 31 01:12:39.994662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:39.991687    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca" pod "86cba2b7-af0f-4a27-889f-cd6b82bc63ad" (UID: "86cba2b7-af0f-4a27-889f-cd6b82bc63ad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:12:40.006638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.005762    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca" pod "86cba2b7-af0f-4a27-889f-cd6b82bc63ad" (UID: "86cba2b7-af0f-4a27-889f-cd6b82bc63ad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:12:40.038826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.038763    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6" pod "86cba2b7-af0f-4a27-889f-cd6b82bc63ad" (UID: "86cba2b7-af0f-4a27-889f-cd6b82bc63ad"). InnerVolumeSpecName "kube-api-access-h5jp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:12:40.086368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.086330    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:40.086368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.086370    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:40.086660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.086386    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5jp6\" (UniqueName: \"kubernetes.io/projected/86cba2b7-af0f-4a27-889f-cd6b82bc63ad-kube-api-access-h5jp6\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:40.445397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.445361    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" event={"ID":"d2813b0a-d570-425a-9523-778af9495993","Type":"ContainerStarted","Data":"0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b"}
Oct 31 01:12:40.446748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.446728    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"
Oct 31 01:12:40.446915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.446754    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67" event={"ID":"86cba2b7-af0f-4a27-889f-cd6b82bc63ad","Type":"ContainerDied","Data":"7de99bad36d2bfe040f774122aa39e96e0ed41b11d5d46b78d5470b76c99a3f7"}
Oct 31 01:12:40.446915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.446789    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7de99bad36d2bfe040f774122aa39e96e0ed41b11d5d46b78d5470b76c99a3f7"
Oct 31 01:12:40.448571 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.448547    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" event={"ID":"93013175-f94f-4939-8d9d-78930d529213","Type":"ContainerStarted","Data":"6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5"}
Oct 31 01:12:40.448747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.448586    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" event={"ID":"93013175-f94f-4939-8d9d-78930d529213","Type":"ContainerStarted","Data":"e5ddcd3bfb54c1fdc5c3bbb6b76ddf8e0f00d9c59f56b3a6aaf71f50c90e28d7"}
Oct 31 01:12:40.467298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.467238    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podStartSLOduration=3.467223991 podStartE2EDuration="3.467223991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:40.463942295 +0000 UTC m=+957.451054271" watchObservedRunningTime="2026-10-31 01:12:40.467223991 +0000 UTC m=+957.454335968"
Oct 31 01:12:40.499591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.499534    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podStartSLOduration=3.499514767 podStartE2EDuration="3.499514767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:40.497233616 +0000 UTC m=+957.484345591" watchObservedRunningTime="2026-10-31 01:12:40.499514767 +0000 UTC m=+957.486626739"
Oct 31 01:12:40.801681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.788948    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:40.846706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.846650    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 31 01:12:40.847104 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.846958    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 31 01:12:40.937631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.936709    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:40.956102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.956063    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 31 01:12:40.956361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:40.956328    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 31 01:12:41.793019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:41.792984    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 31 01:12:41.793661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:41.793629    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 31 01:12:41.912626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:41.912398    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 31 01:12:41.912626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:41.912455    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 31 01:12:42.796628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:42.795000    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 31 01:12:42.796628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:42.795069    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 31 01:12:42.902628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:42.902099    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 31 01:12:42.902628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:42.902163    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 31 01:12:43.790627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:43.789064    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 31 01:12:43.790627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:43.789122    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 31 01:12:43.903968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:43.903933    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 31 01:12:43.904565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:43.904534    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 31 01:12:44.798625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:44.797982    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 31 01:12:44.798625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:44.798041    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 31 01:12:44.914627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:44.904919    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 31 01:12:44.914627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:44.904977    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 31 01:12:45.799628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:45.795199    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 31 01:12:45.799628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:45.795274    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 31 01:12:45.907627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:45.906948    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 31 01:12:45.907627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:45.907035    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 31 01:12:47.197731 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:47.197698    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:47.198221 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:47.198191    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:47.201984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:47.201836    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:12:47.231179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:47.231146    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:47.231674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:47.231625    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:47.232674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:47.232651    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:12:49.452424 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:12:49.452385    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:12:51.688221 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.687943    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"]
Oct 31 01:12:51.688997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.688976    8029 cpu_manager.go:414] "containerMap: removing container" podUID="86cba2b7-af0f-4a27-889f-cd6b82bc63ad" containerName="revoke-keys"
Oct 31 01:12:51.688997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.688998    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86cba2b7-af0f-4a27-889f-cd6b82bc63ad" containerName="revoke-keys"
Oct 31 01:12:51.689188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.689174    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="86cba2b7-af0f-4a27-889f-cd6b82bc63ad" containerName="revoke-keys"
Oct 31 01:12:51.702296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.702268    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.714930 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.714862    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"]
Oct 31 01:12:51.720218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.720172    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"]
Oct 31 01:12:51.744955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.739353    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"]
Oct 31 01:12:51.744955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.740414    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.826720    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.826780    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqnm9\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.826816    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.826846    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.826908    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827057    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827101    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827135    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827162    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2c2v\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827189    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827243    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827373    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827400    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827427    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827451    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827503    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827536    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.828626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.827616    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929126    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929163    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2c2v\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929182    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929221    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929294    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929320    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929343    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929371    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929418    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929449    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929501    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929544    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929583    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqnm9\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929629    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929658    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929723    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.929740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929734    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.930194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929875    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.930194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929909    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.930194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.929958    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.930194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930000    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.930194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930169    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.930471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930267    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.930471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930356    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.930471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930373    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.930825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930526    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.930825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930630    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.930825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.930696    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.932166 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.932144    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.932444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.932425    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.932574 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.932556    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.933236 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.933219    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.936895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.936875    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:51.937081 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.937058    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2c2v\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.937124 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.937085    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:51.937706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:51.937683    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqnm9\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9\") pod \"e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") " pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:52.030781 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.030709    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"]
Oct 31 01:12:52.036845 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.036796    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.040326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.040084    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:12:52.040326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.040190    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:12:52.040326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.040299    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:12:52.040569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.040452    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:12:52.042811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.042791    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:52.044292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.044268    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"]
Oct 31 01:12:52.075770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.075731    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:52.132983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.132557    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.132983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.132697    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.132983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.132864    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmsns\" (UniqueName: \"kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.234769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.234734    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmsns\" (UniqueName: \"kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.235009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.234878    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.235009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.234991    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.236475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.236429    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.239689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.239659    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.244140 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.244111    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmsns\" (UniqueName: \"kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns\") pod \"maas-api-revoke-sub-913e10f0-jb7mb\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.350736 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:52.350707    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda043128b_3095_44ca_877b_173faa3a2099.slice/crio-c6ebecb0b03dffcab62869681d6c5f2939fc6ff4fa23c3010d0ea19789fc42f9 WatchSource:0}: Error finding container c6ebecb0b03dffcab62869681d6c5f2939fc6ff4fa23c3010d0ea19789fc42f9: Status 404 returned error can't find the container with id c6ebecb0b03dffcab62869681d6c5f2939fc6ff4fa23c3010d0ea19789fc42f9
Oct 31 01:12:52.350878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.350773    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"]
Oct 31 01:12:52.353278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.353250    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:52.353360 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.353307    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:52.353360 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.353346    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:52.359465 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.359446    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:52.389507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.389479    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"]
Oct 31 01:12:52.396161 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:52.396139    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c899ad_9c8f_478e_a1f9_101d57b4098f.slice/crio-3dcb92a0c9acc4e56d0282a01b33c2821d56dc10c0d72bd9f6bc18ed33022840 WatchSource:0}: Error finding container 3dcb92a0c9acc4e56d0282a01b33c2821d56dc10c0d72bd9f6bc18ed33022840: Status 404 returned error can't find the container with id 3dcb92a0c9acc4e56d0282a01b33c2821d56dc10c0d72bd9f6bc18ed33022840
Oct 31 01:12:52.399507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.399471    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:52.399592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.399572    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:52.399659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.399642    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:12:52.633102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.633067    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" event={"ID":"d6c899ad-9c8f-478e-a1f9-101d57b4098f","Type":"ContainerStarted","Data":"bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c"}
Oct 31 01:12:52.633310 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.633143    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" event={"ID":"d6c899ad-9c8f-478e-a1f9-101d57b4098f","Type":"ContainerStarted","Data":"3dcb92a0c9acc4e56d0282a01b33c2821d56dc10c0d72bd9f6bc18ed33022840"}
Oct 31 01:12:52.635272 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.635245    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" event={"ID":"a043128b-3095-44ca-877b-173faa3a2099","Type":"ContainerStarted","Data":"d75dde75e6f4dfd8bfcabcb0e88cd9e1714d1a2a0d795c07e2dde57333202c56"}
Oct 31 01:12:52.635371 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.635291    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" event={"ID":"a043128b-3095-44ca-877b-173faa3a2099","Type":"ContainerStarted","Data":"c6ebecb0b03dffcab62869681d6c5f2939fc6ff4fa23c3010d0ea19789fc42f9"}
Oct 31 01:12:52.657486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.655236    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podStartSLOduration=1.655222952 podStartE2EDuration="1.655222952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:52.650866962 +0000 UTC m=+969.637978959" watchObservedRunningTime="2026-10-31 01:12:52.655222952 +0000 UTC m=+969.642334928"
Oct 31 01:12:52.663880 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.662293    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"]
Oct 31 01:12:52.684028 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:52.683968    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podStartSLOduration=1.683950024 podStartE2EDuration="1.683950024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:52.674744398 +0000 UTC m=+969.661856376" watchObservedRunningTime="2026-10-31 01:12:52.683950024 +0000 UTC m=+969.671062000"
Oct 31 01:12:53.043492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.043459    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:12:53.045248 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.045221    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:53.045365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.045269    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:53.076330 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.076300    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:12:53.107777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.107737    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:53.108066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.108036    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:12:53.644173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.644134    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb" event={"ID":"de8f2059-f99c-47bf-8a1c-132b2dc4c63c","Type":"ContainerStarted","Data":"402cecd08a5f5dc2a1f755463a9f2695da1bd88fb57382cd5580527f3575c24c"}
Oct 31 01:12:53.644364 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.644207    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb" event={"ID":"de8f2059-f99c-47bf-8a1c-132b2dc4c63c","Type":"ContainerStarted","Data":"5db453fe264b5788d995401bb7eac6f3975ad4c6c266cd8366a459170477404d"}
Oct 31 01:12:53.729626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:53.728445    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb" podStartSLOduration=1.728422989 podStartE2EDuration="1.728422989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:53.691884012 +0000 UTC m=+970.678995989" watchObservedRunningTime="2026-10-31 01:12:53.728422989 +0000 UTC m=+970.715534965"
Oct 31 01:12:54.043914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:54.043883    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:54.044310 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:54.043934    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:54.076484 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:54.076459    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:54.076639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:54.076510    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:12:54.653774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:54.653738    8029 generic.go:356] "Generic (PLEG): container finished" podID="de8f2059-f99c-47bf-8a1c-132b2dc4c63c" containerID="402cecd08a5f5dc2a1f755463a9f2695da1bd88fb57382cd5580527f3575c24c" exitCode=0
Oct 31 01:12:54.653964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:54.653830    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb" event={"ID":"de8f2059-f99c-47bf-8a1c-132b2dc4c63c","Type":"ContainerDied","Data":"402cecd08a5f5dc2a1f755463a9f2695da1bd88fb57382cd5580527f3575c24c"}
Oct 31 01:12:55.044214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.044130    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:55.044214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.044180    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:55.076364 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.076338    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:55.076518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.076385    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:12:55.750468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.750420    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"]
Oct 31 01:12:55.762011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.758892    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.762011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.761505    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"]
Oct 31 01:12:55.864143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.861529    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qmzx\" (UniqueName: \"kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.864143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.861655    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.864143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.861686    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.963153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.963036    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qmzx\" (UniqueName: \"kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.963233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.963163    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.963233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.963198    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.964911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.964165    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.966791 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.966769    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:55.972185 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:55.972163    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qmzx\" (UniqueName: \"kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx\") pod \"maas-api-revoke-sub-e52cb5f6-t5j4w\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:56.042557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.042531    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:56.043064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.043045    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:56.043118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.043088    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:56.064832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.064801    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns\" (UniqueName: \"kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns\") pod \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") "
Oct 31 01:12:56.065298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.065014    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca\") pod \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") "
Oct 31 01:12:56.065298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.065163    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca\") pod \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\" (UID: \"de8f2059-f99c-47bf-8a1c-132b2dc4c63c\") "
Oct 31 01:12:56.065405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.065298    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca" pod "de8f2059-f99c-47bf-8a1c-132b2dc4c63c" (UID: "de8f2059-f99c-47bf-8a1c-132b2dc4c63c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:12:56.065814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.065791    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:56.067472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.067443    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca" pod "de8f2059-f99c-47bf-8a1c-132b2dc4c63c" (UID: "de8f2059-f99c-47bf-8a1c-132b2dc4c63c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:12:56.069058 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.069039    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns" pod "de8f2059-f99c-47bf-8a1c-132b2dc4c63c" (UID: "de8f2059-f99c-47bf-8a1c-132b2dc4c63c"). InnerVolumeSpecName "kube-api-access-cmsns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:12:56.076482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.076186    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:56.076482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.076244    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:12:56.141636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.140662    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:12:56.167943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.167909    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:56.168058 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.167949    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmsns\" (UniqueName: \"kubernetes.io/projected/de8f2059-f99c-47bf-8a1c-132b2dc4c63c-kube-api-access-cmsns\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:12:56.527497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.527342    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"]
Oct 31 01:12:56.541661 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:12:56.537410    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84812af_980f_430f_b321_31787969eb84.slice/crio-b95a609822729584a6f9b78a756e7e9624fbd8b2d9fdcde84636917cdb2bc1fc WatchSource:0}: Error finding container b95a609822729584a6f9b78a756e7e9624fbd8b2d9fdcde84636917cdb2bc1fc: Status 404 returned error can't find the container with id b95a609822729584a6f9b78a756e7e9624fbd8b2d9fdcde84636917cdb2bc1fc
Oct 31 01:12:56.669797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.669768    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"
Oct 31 01:12:56.669986 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.669766    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb" event={"ID":"de8f2059-f99c-47bf-8a1c-132b2dc4c63c","Type":"ContainerDied","Data":"5db453fe264b5788d995401bb7eac6f3975ad4c6c266cd8366a459170477404d"}
Oct 31 01:12:56.669986 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.669909    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5db453fe264b5788d995401bb7eac6f3975ad4c6c266cd8366a459170477404d"
Oct 31 01:12:56.671135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:56.671107    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w" event={"ID":"e84812af-980f-430f-b321-31787969eb84","Type":"ContainerStarted","Data":"b95a609822729584a6f9b78a756e7e9624fbd8b2d9fdcde84636917cdb2bc1fc"}
Oct 31 01:12:57.046437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:57.046412    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:57.046658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:57.046628    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:57.076168 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:57.076143    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:57.076536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:57.076203    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:12:57.741978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:57.741941    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w" event={"ID":"e84812af-980f-430f-b321-31787969eb84","Type":"ContainerStarted","Data":"5b3b63e19c287e34d297a1ce82ef04136c26452234e27e1c92e0cc866bc8c1d6"}
Oct 31 01:12:57.761576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:57.761529    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w" podStartSLOduration=2.761515466 podStartE2EDuration="2.761515466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:12:57.757433565 +0000 UTC m=+974.744545545" watchObservedRunningTime="2026-10-31 01:12:57.761515466 +0000 UTC m=+974.748627442"
Oct 31 01:12:58.043679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:58.043560    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:58.043679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:58.043636    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:58.076515 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:58.076484    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:58.077024 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:58.076541    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:12:58.754836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:58.754795    8029 generic.go:356] "Generic (PLEG): container finished" podID="e84812af-980f-430f-b321-31787969eb84" containerID="5b3b63e19c287e34d297a1ce82ef04136c26452234e27e1c92e0cc866bc8c1d6" exitCode=0
Oct 31 01:12:58.755016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:58.754873    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w" event={"ID":"e84812af-980f-430f-b321-31787969eb84","Type":"ContainerDied","Data":"5b3b63e19c287e34d297a1ce82ef04136c26452234e27e1c92e0cc866bc8c1d6"}
Oct 31 01:12:59.044280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:59.044177    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 31 01:12:59.044280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:59.044248    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 31 01:12:59.076500 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:59.076469    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 31 01:12:59.076712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:12:59.076521    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 31 01:13:00.030329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.030299    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:13:00.100741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.100711    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:13:00.101714 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.101690    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:13:00.103270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.102919    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:13:00.106850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.106827    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:13:00.108043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.108027    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:13:00.111045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.109809    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:13:00.111744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.111680    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx\" (UniqueName: \"kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx\") pod \"e84812af-980f-430f-b321-31787969eb84\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") "
Oct 31 01:13:00.111892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.111805    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca\") pod \"e84812af-980f-430f-b321-31787969eb84\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") "
Oct 31 01:13:00.111892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.111846    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca\") pod \"e84812af-980f-430f-b321-31787969eb84\" (UID: \"e84812af-980f-430f-b321-31787969eb84\") "
Oct 31 01:13:00.112778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.112566    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca" pod "e84812af-980f-430f-b321-31787969eb84" (UID: "e84812af-980f-430f-b321-31787969eb84"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:00.112778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.112719    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e84812af-980f-430f-b321-31787969eb84-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:00.114729 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.114664    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca" pod "e84812af-980f-430f-b321-31787969eb84" (UID: "e84812af-980f-430f-b321-31787969eb84"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:00.115813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.115784    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx" pod "e84812af-980f-430f-b321-31787969eb84" (UID: "e84812af-980f-430f-b321-31787969eb84"). InnerVolumeSpecName "kube-api-access-4qmzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:00.215040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.215007    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qmzx\" (UniqueName: \"kubernetes.io/projected/e84812af-980f-430f-b321-31787969eb84-kube-api-access-4qmzx\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:00.215273 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.215251    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e84812af-980f-430f-b321-31787969eb84-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:00.656755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.656721    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"]
Oct 31 01:13:00.657461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.657443    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e84812af-980f-430f-b321-31787969eb84" containerName="revoke-keys"
Oct 31 01:13:00.657461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.657461    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e84812af-980f-430f-b321-31787969eb84" containerName="revoke-keys"
Oct 31 01:13:00.657615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.657472    8029 cpu_manager.go:414] "containerMap: removing container" podUID="de8f2059-f99c-47bf-8a1c-132b2dc4c63c" containerName="revoke-keys"
Oct 31 01:13:00.657615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.657477    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de8f2059-f99c-47bf-8a1c-132b2dc4c63c" containerName="revoke-keys"
Oct 31 01:13:00.657682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.657632    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e84812af-980f-430f-b321-31787969eb84" containerName="revoke-keys"
Oct 31 01:13:00.657682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.657642    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="de8f2059-f99c-47bf-8a1c-132b2dc4c63c" containerName="revoke-keys"
Oct 31 01:13:00.661423 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.661399    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.669074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.669048    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"]
Oct 31 01:13:00.775685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.775656    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"
Oct 31 01:13:00.775685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.775662    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w" event={"ID":"e84812af-980f-430f-b321-31787969eb84","Type":"ContainerDied","Data":"b95a609822729584a6f9b78a756e7e9624fbd8b2d9fdcde84636917cdb2bc1fc"}
Oct 31 01:13:00.775901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.775718    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b95a609822729584a6f9b78a756e7e9624fbd8b2d9fdcde84636917cdb2bc1fc"
Oct 31 01:13:00.823939 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.823905    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xldpj\" (UniqueName: \"kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.824126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.823995    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.824176 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.824167    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.926223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.926044    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldpj\" (UniqueName: \"kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.926223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.926149    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.926585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.926339    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.927055 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.927037    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.928680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.928663    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.933935 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.933911    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xldpj\" (UniqueName: \"kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj\") pod \"maas-api-revoke-sub-76d9dc0b-xqqd5\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:00.980164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:00.980130    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:01.272118 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:01.272089    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bbb45f_ddbc_44c4_a5b7_e9e4d7e2fa12.slice/crio-3e1546142ce53cfee7510d25a22df0471f5c470c05b94a05e1afccec5942ed8e WatchSource:0}: Error finding container 3e1546142ce53cfee7510d25a22df0471f5c470c05b94a05e1afccec5942ed8e: Status 404 returned error can't find the container with id 3e1546142ce53cfee7510d25a22df0471f5c470c05b94a05e1afccec5942ed8e
Oct 31 01:13:01.272956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:01.272930    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"]
Oct 31 01:13:01.786520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:01.786486    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5" event={"ID":"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12","Type":"ContainerStarted","Data":"ef65f0b77ef15fd0819b35edba37d4f2d2016b8fa3f9a3f44a73a9992f73de17"}
Oct 31 01:13:01.786756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:01.786542    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5" event={"ID":"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12","Type":"ContainerStarted","Data":"3e1546142ce53cfee7510d25a22df0471f5c470c05b94a05e1afccec5942ed8e"}
Oct 31 01:13:01.805621 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:01.805555    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5" podStartSLOduration=1.805541851 podStartE2EDuration="1.805541851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:01.802175463 +0000 UTC m=+978.789287449" watchObservedRunningTime="2026-10-31 01:13:01.805541851 +0000 UTC m=+978.792653826"
Oct 31 01:13:02.797302 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:02.797265    8029 generic.go:356] "Generic (PLEG): container finished" podID="56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" containerID="ef65f0b77ef15fd0819b35edba37d4f2d2016b8fa3f9a3f44a73a9992f73de17" exitCode=0
Oct 31 01:13:02.797786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:02.797351    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5" event={"ID":"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12","Type":"ContainerDied","Data":"ef65f0b77ef15fd0819b35edba37d4f2d2016b8fa3f9a3f44a73a9992f73de17"}
Oct 31 01:13:04.866736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:04.864676    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:04.912627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:04.911928    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5" event={"ID":"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12","Type":"ContainerDied","Data":"3e1546142ce53cfee7510d25a22df0471f5c470c05b94a05e1afccec5942ed8e"}
Oct 31 01:13:04.912627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:04.911992    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e1546142ce53cfee7510d25a22df0471f5c470c05b94a05e1afccec5942ed8e"
Oct 31 01:13:04.912627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:04.912065    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"
Oct 31 01:13:05.009504 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.005528    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj\" (UniqueName: \"kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj\") pod \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") "
Oct 31 01:13:05.009504 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.005640    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca\") pod \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") "
Oct 31 01:13:05.009504 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.005801    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca\") pod \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\" (UID: \"56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12\") "
Oct 31 01:13:05.054295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.054252    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca" pod "56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" (UID: "56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:05.110237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.107729    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca" pod "56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" (UID: "56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:05.110237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.108065    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:05.110237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.108094    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:05.110237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.108206    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj" pod "56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" (UID: "56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12"). InnerVolumeSpecName "kube-api-access-xldpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:05.235226 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.235190    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xldpj\" (UniqueName: \"kubernetes.io/projected/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12-kube-api-access-xldpj\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:05.873689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:05.871767    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:06.065638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.056742    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:06.090628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.072692    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:06.104738 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.104688    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:06.140815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.140702    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:06.285268 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.282736    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:06.353471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.353426    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:06.473781 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:13:06.468585    8029 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bbb45f_ddbc_44c4_a5b7_e9e4d7e2fa12.slice\": RecentStats: unable to find data in memory cache]"
Oct 31 01:13:06.931642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.901043    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:06.931642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:06.901646    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:13:07.076850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.076756    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:07.077137 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.077111    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:07.095679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.077379    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:13:07.209395 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.119772    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:07.354044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.144263    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:07.354898 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.209674    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"]
Oct 31 01:13:07.356093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.356066    8029 cpu_manager.go:414] "containerMap: removing container" podUID="56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" containerName="revoke-keys"
Oct 31 01:13:07.461293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.461209    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" containerName="revoke-keys"
Oct 31 01:13:07.464971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.464939    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" containerName="revoke-keys"
Oct 31 01:13:07.478401 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.243741    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:07.498914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.306930    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:07.512316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.362877    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:07.583027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.582984    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:13:07.583193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.583036    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"]
Oct 31 01:13:07.595881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.595850    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.761686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.703621    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.904722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.904680    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.904986 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.904962    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57xp\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.908628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.905160    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.908857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.908832    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.909004 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.908986    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.909253 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.909233    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:07.909407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:07.909390    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.044224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.044130    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.044890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.011674    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:08.120677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.098704    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:08.120677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.100096    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:13:08.120677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.100152    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:13:08.120677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.100187    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:13:08.120677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.100366    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:13:08.178578 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.178541    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.178832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.178814    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.179016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.178989    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.179197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.179159    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.179514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.179485    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.179653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.179637    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57xp\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.179820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.179790    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.224631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.193590    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.224631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.193690    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.236627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.225295    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.236627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.180559    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.236627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.225593    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.236627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.225782    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.321789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.085865    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:08.321789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.306233    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.362789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.335764    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.434948 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.363148    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" podUID="eeda0349-fac8-42fc-a73d-f81c06888419" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.24:6080/livez\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:08.435526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.366239    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:08.473205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.366273    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:08.530972 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.530930    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:08.594013 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.592812    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:08.669629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.662193    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.695091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.695055    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57xp\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.730966 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.730881    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token\") pod \"e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") " pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:08.807629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:08.790978    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:09.173680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.060166    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:09.173680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.121758    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:09.173680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.147750    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" podUID="eeda0349-fac8-42fc-a73d-f81c06888419" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.24:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:09.367645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.362931    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:09.428301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.428197    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/health\": context deadline exceeded"
Oct 31 01:13:09.501181 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.501138    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:09.501362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.501168    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:09.542931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.542868    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:09.608497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.598778    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:10.003622 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:09.995713    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded"
Oct 31 01:13:10.075447 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.075053    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:10.149661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.145838    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:10.368499 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.364762    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:10.450883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.450841    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"]
Oct 31 01:13:10.508119 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.508077    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"]
Oct 31 01:13:10.508394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.508372    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"]
Oct 31 01:13:10.542823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.509120    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:10.542823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.512133    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.543587    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.566646    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.566736    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.566952    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.567013    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.567093    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.576533    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.588745    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.589307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.589101    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrsz6\" (UniqueName: \"kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.626094 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.626019    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:10.723956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.704854    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrsz6\" (UniqueName: \"kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.723956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.704951    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.723956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.704999    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.723956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.705955    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"]
Oct 31 01:13:10.723956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.706059    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.723956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.723882    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:13:10.764760 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.763766    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:13:10.766503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.764993    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:13:10.767136 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.766884    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:13:10.769669 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.768688    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrsz6\" (UniqueName: \"kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.769669 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.768738    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.771161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.770376    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:13:10.771161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.770493    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:13:10.799790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.799755    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:13:10.815874 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.814505    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6d34d949-8z68p\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:10.815874 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.815540    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2r2x\" (UniqueName: \"kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.815874 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.815705    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.817628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.816220    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.918464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.918378    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.919090 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.919069    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2r2x\" (UniqueName: \"kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.919373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.919342    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.919482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.919463    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.922651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.922630    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.936686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.935014    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2r2x\" (UniqueName: \"kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x\") pod \"maas-api-revoke-sub-183fb75c-mmdd2\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:10.936686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:10.936479    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:11.005160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.004974    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"]
Oct 31 01:13:11.007716 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:11.007668    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb32cdd_844b_4317_83b7_14b771223847.slice/crio-35a5da4e8a625ddd4645b3340c0f1559a1513c64199c5c7900e612daafeebf0e WatchSource:0}: Error finding container 35a5da4e8a625ddd4645b3340c0f1559a1513c64199c5c7900e612daafeebf0e: Status 404 returned error can't find the container with id 35a5da4e8a625ddd4645b3340c0f1559a1513c64199c5c7900e612daafeebf0e
Oct 31 01:13:11.010769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.010417    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:11.010769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.010479    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:11.010769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.010507    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:11.102942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.102640    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:11.311656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.311614    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"]
Oct 31 01:13:11.312143 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:11.312111    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05fafde_53c9_4a5e_91f3_5659c551add2.slice/crio-cf9800fe9ebed30a8a0c8ce30a08c0c95a2ec8587782912670d4d69e0eb0caa1 WatchSource:0}: Error finding container cf9800fe9ebed30a8a0c8ce30a08c0c95a2ec8587782912670d4d69e0eb0caa1: Status 404 returned error can't find the container with id cf9800fe9ebed30a8a0c8ce30a08c0c95a2ec8587782912670d4d69e0eb0caa1
Oct 31 01:13:11.427944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.427912    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"]
Oct 31 01:13:11.454673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.454555    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p" event={"ID":"f05fafde-53c9-4a5e-91f3-5659c551add2","Type":"ContainerStarted","Data":"cf9800fe9ebed30a8a0c8ce30a08c0c95a2ec8587782912670d4d69e0eb0caa1"}
Oct 31 01:13:11.456814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.456781    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" event={"ID":"5eb32cdd-844b-4317-83b7-14b771223847","Type":"ContainerStarted","Data":"b9b02a962f0b65dbfc1715a7db65858fb78df6e308c344b1da41ac1f877f7913"}
Oct 31 01:13:11.456929 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.456837    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" event={"ID":"5eb32cdd-844b-4317-83b7-14b771223847","Type":"ContainerStarted","Data":"35a5da4e8a625ddd4645b3340c0f1559a1513c64199c5c7900e612daafeebf0e"}
Oct 31 01:13:11.483647 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:11.483588    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5752fcac_a881_4c0e_baac_37d39a2fce9e.slice/crio-d96a5dcc67512b8dd40d2bfac09501eb0de86585d493bfffa76c9895a67ef9df WatchSource:0}: Error finding container d96a5dcc67512b8dd40d2bfac09501eb0de86585d493bfffa76c9895a67ef9df: Status 404 returned error can't find the container with id d96a5dcc67512b8dd40d2bfac09501eb0de86585d493bfffa76c9895a67ef9df
Oct 31 01:13:11.489267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.489209    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podStartSLOduration=5.489187892 podStartE2EDuration="5.489187892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:11.480409083 +0000 UTC m=+988.467521060" watchObservedRunningTime="2026-10-31 01:13:11.489187892 +0000 UTC m=+988.476299869"
Oct 31 01:13:11.797881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:11.797843    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:12.519337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.519010    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2" event={"ID":"5752fcac-a881-4c0e-baac-37d39a2fce9e","Type":"ContainerStarted","Data":"505e172248f77bd64b53ec086b2d44612b4c086e2c673567bf65c952f953393c"}
Oct 31 01:13:12.519337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.519124    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2" event={"ID":"5752fcac-a881-4c0e-baac-37d39a2fce9e","Type":"ContainerStarted","Data":"d96a5dcc67512b8dd40d2bfac09501eb0de86585d493bfffa76c9895a67ef9df"}
Oct 31 01:13:12.523674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.523484    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p" event={"ID":"f05fafde-53c9-4a5e-91f3-5659c551add2","Type":"ContainerStarted","Data":"c57e64fec6639ffba49a8808879135ae95cf2bdabd3e446b3c605a9f422e9ec1"}
Oct 31 01:13:12.550053 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:13:12.539217    8029 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05fafde_53c9_4a5e_91f3_5659c551add2.slice/crio-conmon-c57e64fec6639ffba49a8808879135ae95cf2bdabd3e446b3c605a9f422e9ec1.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05fafde_53c9_4a5e_91f3_5659c551add2.slice/crio-c57e64fec6639ffba49a8808879135ae95cf2bdabd3e446b3c605a9f422e9ec1.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 01:13:12.550053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.544403    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2" podStartSLOduration=2.5443909319999998 podStartE2EDuration="2.544390932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:12.539690999 +0000 UTC m=+989.526802975" watchObservedRunningTime="2026-10-31 01:13:12.544390932 +0000 UTC m=+989.531502933"
Oct 31 01:13:12.733987 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.733951    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"]
Oct 31 01:13:12.738560 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.738538    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:12.752331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.751436    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"]
Oct 31 01:13:12.812732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.807778    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 01:13:12.812732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.807836    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 01:13:12.911186 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.911127    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k65m5\" (UniqueName: \"kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:12.911361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.911270    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:12.911503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:12.911406    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.012889 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.012854    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.013092 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.013007    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65m5\" (UniqueName: \"kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.013092 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.013061    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.013746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.013713    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.015641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.015575    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.024797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.024776    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k65m5\" (UniqueName: \"kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5\") pod \"maas-api-revoke-sub-0f82041d-6b7sg\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.057057 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.057029    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:13.413089 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:13.413048    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd87eef85_111c_4ed6_afd9_3245cb0929e5.slice/crio-81b99933ed595a741d00bf2f001d93591d490afa653fdabcf2a53ab98be93972 WatchSource:0}: Error finding container 81b99933ed595a741d00bf2f001d93591d490afa653fdabcf2a53ab98be93972: Status 404 returned error can't find the container with id 81b99933ed595a741d00bf2f001d93591d490afa653fdabcf2a53ab98be93972
Oct 31 01:13:13.414196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.414054    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"]
Oct 31 01:13:13.532833 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.532793    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg" event={"ID":"d87eef85-111c-4ed6-afd9-3245cb0929e5","Type":"ContainerStarted","Data":"81b99933ed595a741d00bf2f001d93591d490afa653fdabcf2a53ab98be93972"}
Oct 31 01:13:13.534924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.534897    8029 generic.go:356] "Generic (PLEG): container finished" podID="f05fafde-53c9-4a5e-91f3-5659c551add2" containerID="c57e64fec6639ffba49a8808879135ae95cf2bdabd3e446b3c605a9f422e9ec1" exitCode=0
Oct 31 01:13:13.535079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.534992    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p" event={"ID":"f05fafde-53c9-4a5e-91f3-5659c551add2","Type":"ContainerDied","Data":"c57e64fec6639ffba49a8808879135ae95cf2bdabd3e446b3c605a9f422e9ec1"}
Oct 31 01:13:13.537347 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.537320    8029 generic.go:356] "Generic (PLEG): container finished" podID="5752fcac-a881-4c0e-baac-37d39a2fce9e" containerID="505e172248f77bd64b53ec086b2d44612b4c086e2c673567bf65c952f953393c" exitCode=0
Oct 31 01:13:13.537538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.537510    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2" event={"ID":"5752fcac-a881-4c0e-baac-37d39a2fce9e","Type":"ContainerDied","Data":"505e172248f77bd64b53ec086b2d44612b4c086e2c673567bf65c952f953393c"}
Oct 31 01:13:13.776368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.776344    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:13.798663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.797053    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 01:13:13.798663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.797106    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 01:13:13.940786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.939304    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca\") pod \"f05fafde-53c9-4a5e-91f3-5659c551add2\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") "
Oct 31 01:13:13.940786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.939444    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6\" (UniqueName: \"kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6\") pod \"f05fafde-53c9-4a5e-91f3-5659c551add2\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") "
Oct 31 01:13:13.940786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.939555    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca\") pod \"f05fafde-53c9-4a5e-91f3-5659c551add2\" (UID: \"f05fafde-53c9-4a5e-91f3-5659c551add2\") "
Oct 31 01:13:13.941800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.941750    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca" pod "f05fafde-53c9-4a5e-91f3-5659c551add2" (UID: "f05fafde-53c9-4a5e-91f3-5659c551add2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:13.955399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.954747    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca" pod "f05fafde-53c9-4a5e-91f3-5659c551add2" (UID: "f05fafde-53c9-4a5e-91f3-5659c551add2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:13.955399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:13.954776    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6" pod "f05fafde-53c9-4a5e-91f3-5659c551add2" (UID: "f05fafde-53c9-4a5e-91f3-5659c551add2"). InnerVolumeSpecName "kube-api-access-wrsz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:14.042887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.042857    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:14.042887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.042888    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f05fafde-53c9-4a5e-91f3-5659c551add2-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:14.043066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.042902    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrsz6\" (UniqueName: \"kubernetes.io/projected/f05fafde-53c9-4a5e-91f3-5659c551add2-kube-api-access-wrsz6\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:14.548135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.548099    8029 generic.go:356] "Generic (PLEG): container finished" podID="d87eef85-111c-4ed6-afd9-3245cb0929e5" containerID="12de93ad7ddbcd4ebd2f652bd601906a11869864f957a34b19daa252ff8b45e2" exitCode=0
Oct 31 01:13:14.548652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.548202    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg" event={"ID":"d87eef85-111c-4ed6-afd9-3245cb0929e5","Type":"ContainerDied","Data":"12de93ad7ddbcd4ebd2f652bd601906a11869864f957a34b19daa252ff8b45e2"}
Oct 31 01:13:14.550173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.550148    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"
Oct 31 01:13:14.550291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.550171    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p" event={"ID":"f05fafde-53c9-4a5e-91f3-5659c551add2","Type":"ContainerDied","Data":"cf9800fe9ebed30a8a0c8ce30a08c0c95a2ec8587782912670d4d69e0eb0caa1"}
Oct 31 01:13:14.550291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.550230    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf9800fe9ebed30a8a0c8ce30a08c0c95a2ec8587782912670d4d69e0eb0caa1"
Oct 31 01:13:14.803099 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.803059    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 01:13:14.803369 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.803341    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 01:13:14.821213 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.821183    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:14.956977 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.956946    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca\") pod \"5752fcac-a881-4c0e-baac-37d39a2fce9e\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") "
Oct 31 01:13:14.957169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.957083    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x\" (UniqueName: \"kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x\") pod \"5752fcac-a881-4c0e-baac-37d39a2fce9e\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") "
Oct 31 01:13:14.957218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.957193    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca\") pod \"5752fcac-a881-4c0e-baac-37d39a2fce9e\" (UID: \"5752fcac-a881-4c0e-baac-37d39a2fce9e\") "
Oct 31 01:13:14.957337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.957311    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca" pod "5752fcac-a881-4c0e-baac-37d39a2fce9e" (UID: "5752fcac-a881-4c0e-baac-37d39a2fce9e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:14.957818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.957795    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:14.959300 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.959272    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x" pod "5752fcac-a881-4c0e-baac-37d39a2fce9e" (UID: "5752fcac-a881-4c0e-baac-37d39a2fce9e"). InnerVolumeSpecName "kube-api-access-l2r2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:14.959414 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:14.959393    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca" pod "5752fcac-a881-4c0e-baac-37d39a2fce9e" (UID: "5752fcac-a881-4c0e-baac-37d39a2fce9e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:15.059716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.059594    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2r2x\" (UniqueName: \"kubernetes.io/projected/5752fcac-a881-4c0e-baac-37d39a2fce9e-kube-api-access-l2r2x\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:15.059716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.059671    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5752fcac-a881-4c0e-baac-37d39a2fce9e-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:15.560760 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.560732    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"
Oct 31 01:13:15.561170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.560770    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2" event={"ID":"5752fcac-a881-4c0e-baac-37d39a2fce9e","Type":"ContainerDied","Data":"d96a5dcc67512b8dd40d2bfac09501eb0de86585d493bfffa76c9895a67ef9df"}
Oct 31 01:13:15.561170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.560820    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d96a5dcc67512b8dd40d2bfac09501eb0de86585d493bfffa76c9895a67ef9df"
Oct 31 01:13:15.763157 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.763129    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:15.810068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.810030    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 01:13:15.810260 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.810106    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 01:13:15.871546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.871517    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca\") pod \"d87eef85-111c-4ed6-afd9-3245cb0929e5\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") "
Oct 31 01:13:15.871728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.871585    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5\" (UniqueName: \"kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5\") pod \"d87eef85-111c-4ed6-afd9-3245cb0929e5\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") "
Oct 31 01:13:15.871837 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.871823    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca\") pod \"d87eef85-111c-4ed6-afd9-3245cb0929e5\" (UID: \"d87eef85-111c-4ed6-afd9-3245cb0929e5\") "
Oct 31 01:13:15.872176 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.872150    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca" pod "d87eef85-111c-4ed6-afd9-3245cb0929e5" (UID: "d87eef85-111c-4ed6-afd9-3245cb0929e5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:15.872380 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.872364    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:15.873878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.873852    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5" pod "d87eef85-111c-4ed6-afd9-3245cb0929e5" (UID: "d87eef85-111c-4ed6-afd9-3245cb0929e5"). InnerVolumeSpecName "kube-api-access-k65m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:15.873878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.873857    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca" pod "d87eef85-111c-4ed6-afd9-3245cb0929e5" (UID: "d87eef85-111c-4ed6-afd9-3245cb0929e5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:15.974187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.974154    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d87eef85-111c-4ed6-afd9-3245cb0929e5-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:15.974187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:15.974183    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k65m5\" (UniqueName: \"kubernetes.io/projected/d87eef85-111c-4ed6-afd9-3245cb0929e5-kube-api-access-k65m5\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:16.571132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:16.571096    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg" event={"ID":"d87eef85-111c-4ed6-afd9-3245cb0929e5","Type":"ContainerDied","Data":"81b99933ed595a741d00bf2f001d93591d490afa653fdabcf2a53ab98be93972"}
Oct 31 01:13:16.571540 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:16.571146    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81b99933ed595a741d00bf2f001d93591d490afa653fdabcf2a53ab98be93972"
Oct 31 01:13:16.571540 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:16.571102    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"
Oct 31 01:13:16.802353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:16.802158    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 01:13:16.802353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:16.802217    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 01:13:17.803625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:17.798989    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 01:13:17.803625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:17.799051    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 01:13:18.801088 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.801019    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:18.801513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.801492    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:18.802212 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.802189    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:13:18.807806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.807785    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"]
Oct 31 01:13:18.808414 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808400    8029 cpu_manager.go:414] "containerMap: removing container" podUID="f05fafde-53c9-4a5e-91f3-5659c551add2" containerName="revoke-keys"
Oct 31 01:13:18.808456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808415    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f05fafde-53c9-4a5e-91f3-5659c551add2" containerName="revoke-keys"
Oct 31 01:13:18.808456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808448    8029 cpu_manager.go:414] "containerMap: removing container" podUID="d87eef85-111c-4ed6-afd9-3245cb0929e5" containerName="revoke-keys"
Oct 31 01:13:18.808456 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808453    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d87eef85-111c-4ed6-afd9-3245cb0929e5" containerName="revoke-keys"
Oct 31 01:13:18.808549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808476    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5752fcac-a881-4c0e-baac-37d39a2fce9e" containerName="revoke-keys"
Oct 31 01:13:18.808549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808481    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5752fcac-a881-4c0e-baac-37d39a2fce9e" containerName="revoke-keys"
Oct 31 01:13:18.808622 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808588    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5752fcac-a881-4c0e-baac-37d39a2fce9e" containerName="revoke-keys"
Oct 31 01:13:18.808622 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808612    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f05fafde-53c9-4a5e-91f3-5659c551add2" containerName="revoke-keys"
Oct 31 01:13:18.808690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.808629    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d87eef85-111c-4ed6-afd9-3245cb0929e5" containerName="revoke-keys"
Oct 31 01:13:18.813031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.813011    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.824915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.824893    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:13:18.825020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.824920    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:13:18.825116 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.825101    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:13:18.833715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.833689    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"]
Oct 31 01:13:18.839221 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.839200    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.839338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.839243    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7vd7\" (UniqueName: \"kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.839437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.839351    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.924314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.924286    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:13:18.942033 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.941999    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.942213 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.942051    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7vd7\" (UniqueName: \"kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.942213 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.942128    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.943123 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.943095    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.944584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.944562    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:18.958883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:18.958860    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7vd7\" (UniqueName: \"kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7\") pod \"maas-api-revoke-sub-dd0ee2b3-hktqf\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:19.128253 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:19.128217    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:19.414096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:19.414070    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"]
Oct 31 01:13:19.414746 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:19.414722    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edab968_fd36_450c_bc2c_1f07648b642d.slice/crio-ed2d27d43b4c978c656f76e3c90dfd0da0b0257d73b8e5dd2c8e9955bd72eb1f WatchSource:0}: Error finding container ed2d27d43b4c978c656f76e3c90dfd0da0b0257d73b8e5dd2c8e9955bd72eb1f: Status 404 returned error can't find the container with id ed2d27d43b4c978c656f76e3c90dfd0da0b0257d73b8e5dd2c8e9955bd72eb1f
Oct 31 01:13:19.447877 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:13:19.447846    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:13:19.607926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:19.607887    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf" event={"ID":"1edab968-fd36-450c-bc2c-1f07648b642d","Type":"ContainerStarted","Data":"ed2d27d43b4c978c656f76e3c90dfd0da0b0257d73b8e5dd2c8e9955bd72eb1f"}
Oct 31 01:13:20.617806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:20.617777    8029 generic.go:356] "Generic (PLEG): container finished" podID="1edab968-fd36-450c-bc2c-1f07648b642d" containerID="d781b70a1d08272680620b926b24d0ad2d437dd772605b583ce70cb750e08215" exitCode=0
Oct 31 01:13:20.618170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:20.617851    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf" event={"ID":"1edab968-fd36-450c-bc2c-1f07648b642d","Type":"ContainerDied","Data":"d781b70a1d08272680620b926b24d0ad2d437dd772605b583ce70cb750e08215"}
Oct 31 01:13:21.793795 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.793769    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:21.883041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.882960    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca\") pod \"1edab968-fd36-450c-bc2c-1f07648b642d\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") "
Oct 31 01:13:21.883041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.883015    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7\" (UniqueName: \"kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7\") pod \"1edab968-fd36-450c-bc2c-1f07648b642d\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") "
Oct 31 01:13:21.883041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.883037    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca\") pod \"1edab968-fd36-450c-bc2c-1f07648b642d\" (UID: \"1edab968-fd36-450c-bc2c-1f07648b642d\") "
Oct 31 01:13:21.883483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.883458    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca" pod "1edab968-fd36-450c-bc2c-1f07648b642d" (UID: "1edab968-fd36-450c-bc2c-1f07648b642d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:21.883899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.883878    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:21.885468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.885446    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca" pod "1edab968-fd36-450c-bc2c-1f07648b642d" (UID: "1edab968-fd36-450c-bc2c-1f07648b642d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:21.885468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.885459    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7" pod "1edab968-fd36-450c-bc2c-1f07648b642d" (UID: "1edab968-fd36-450c-bc2c-1f07648b642d"). InnerVolumeSpecName "kube-api-access-k7vd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:21.918885 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.918854    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"]
Oct 31 01:13:21.920100 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.920074    8029 cpu_manager.go:414] "containerMap: removing container" podUID="1edab968-fd36-450c-bc2c-1f07648b642d" containerName="revoke-keys"
Oct 31 01:13:21.920249 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.920235    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1edab968-fd36-450c-bc2c-1f07648b642d" containerName="revoke-keys"
Oct 31 01:13:21.920525 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.920513    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="1edab968-fd36-450c-bc2c-1f07648b642d" containerName="revoke-keys"
Oct 31 01:13:21.926307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.926272    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.950499 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.945977    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"]
Oct 31 01:13:21.966586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.966557    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"]
Oct 31 01:13:21.966757 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.966704    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:21.974883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.970946    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"]
Oct 31 01:13:21.985088 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985057    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985204    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985356 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985333    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985380    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985427    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985561 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985543    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985671 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985637    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985676    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msq5x\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985740    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:21.985854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985843    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1edab968-fd36-450c-bc2c-1f07648b642d-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:21.985904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:21.985861    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7vd7\" (UniqueName: \"kubernetes.io/projected/1edab968-fd36-450c-bc2c-1f07648b642d-kube-api-access-k7vd7\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:22.087787 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.087746    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.087952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.087865    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmtrj\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.087952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.087899    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.087952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.087925    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.087954    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.087996    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088050    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088097    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088229    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088273    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088448    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088505    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088555 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088539    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msq5x\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088627    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088664    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088695    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.088777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088763    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.088831 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.088817    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.089098 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.089076    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.089389 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.089362    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.090040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.089996    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.090286 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.090261    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.090488 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.090465    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.092226 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.092204    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.092375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.092302    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.097968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.097936    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.098284 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.098259    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msq5x\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.189755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.189671    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmtrj\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.189755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.189708    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.189755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.189728    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.189974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.189788    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.189974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.189906    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.189981    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190049    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190102    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190114    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190148    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190223    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190412 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190393    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190452    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.190523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.190499    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.192514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.192484    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.192691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.192676    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.201918 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.201851    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmtrj\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.202134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.202112    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token\") pod \"e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") " pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.245509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.245471    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"]
Oct 31 01:13:22.252167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.252139    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.262588 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.262462    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"]
Oct 31 01:13:22.268945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.268489    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:22.268945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.268790    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.281978    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"]
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.291757    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.291816    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.291912    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfcp\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292042    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292114    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292152    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292210    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292249    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.292505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292322    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.293148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.292564    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:22.293979 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.293888    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"]
Oct 31 01:13:22.328890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.324554    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"]
Oct 31 01:13:22.332231 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.331785    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.337785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.337763    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"]
Oct 31 01:13:22.393736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393682    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.393736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393748    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393781    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393811    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393832    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393855    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393881    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393898    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393923    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.393977    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394023    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394055    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394079    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394099    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x855v\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394118    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.394159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394150    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfcp\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394170    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69cm7\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394233    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394269    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394288    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394307    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394367    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394668    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394745    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394811    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394886    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.394956    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.395237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.395131    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.399712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.395377    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.399712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.395687    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.399712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.395947    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.399712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.395965    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.399712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.398755    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.404435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.403957    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.404435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.404398    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfcp\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.427453 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.427419    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy\") pod \"e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") " pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.438164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.438134    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"]
Oct 31 01:13:22.445084 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.444921    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.459126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.459091    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"]
Oct 31 01:13:22.497664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497629    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.497819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497695    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.497819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497731    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x855v\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.497819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497762    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.497931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497813    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69cm7\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.497931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497884    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.497931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497914    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.498020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497951    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.498020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.497978    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.498020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498004    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.498110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498042    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.498110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498053    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498153    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498249 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498229    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.498319 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498304    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498421 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498408    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498476 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498456    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.498508 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498457    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd92b\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498528    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498590    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.498697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498683    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.498750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498735    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.498808 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498789    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.498808 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498797    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.498875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.498819    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.499069 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.499049    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.500215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.499799    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.500215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.499918    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.500755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.500731    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.501361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.501336    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.501513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.501495    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.501618 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.501529    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.501618 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.501589    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.501731 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.501635    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.502018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.501995    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.502310 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.502288    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.502396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.502374    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.502556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.502534    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.503464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.503444    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.503798 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.503780    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.505914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.505874    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.508404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.508372    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.509255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.509228    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x855v\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.509347 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.509328    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69cm7\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.509665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.509576    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token\") pod \"e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") " pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.603657 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603580    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.603796 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603698    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.603796 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603740    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.603876 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603823    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.603941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603898    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.603941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603936    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd92b\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.603963    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.604017    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604097 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.604088    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604541 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.604506    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.604583    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.604814    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.604915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.604826    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.605189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.605165    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.606656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.606621    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.606741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.606636    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.612425 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.612397    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.612537 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.612521    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd92b\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b\") pod \"e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") " pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.621883 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:22.621849    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb2044d_95c3_409b_862d_d4758727f4bb.slice/crio-9c524aa8bb8bd14e1c8b757c46f9354259a7c93b722b9e74ace7c5eba01955ea WatchSource:0}: Error finding container 9c524aa8bb8bd14e1c8b757c46f9354259a7c93b722b9e74ace7c5eba01955ea: Status 404 returned error can't find the container with id 9c524aa8bb8bd14e1c8b757c46f9354259a7c93b722b9e74ace7c5eba01955ea
Oct 31 01:13:22.622271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.622253    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"]
Oct 31 01:13:22.625285 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.625255    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:22.625382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.625316    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:22.625382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.625344    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:22.629487 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.629463    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"]
Oct 31 01:13:22.629771 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:22.629740    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328e2256_e817_4994_90c8_4c27a7c1f5ab.slice/crio-84c66f4f4b2660c8038052d7a9ba9cc4ba442cd1651e1901f9f55bb27ed13041 WatchSource:0}: Error finding container 84c66f4f4b2660c8038052d7a9ba9cc4ba442cd1651e1901f9f55bb27ed13041: Status 404 returned error can't find the container with id 84c66f4f4b2660c8038052d7a9ba9cc4ba442cd1651e1901f9f55bb27ed13041
Oct 31 01:13:22.632857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.632839    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:22.634148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.634126    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" event={"ID":"5cb2044d-95c3-409b-862d-d4758727f4bb","Type":"ContainerStarted","Data":"9c524aa8bb8bd14e1c8b757c46f9354259a7c93b722b9e74ace7c5eba01955ea"}
Oct 31 01:13:22.635627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.635577    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"
Oct 31 01:13:22.635740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.635671    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf" event={"ID":"1edab968-fd36-450c-bc2c-1f07648b642d","Type":"ContainerDied","Data":"ed2d27d43b4c978c656f76e3c90dfd0da0b0257d73b8e5dd2c8e9955bd72eb1f"}
Oct 31 01:13:22.635740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.635720    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed2d27d43b4c978c656f76e3c90dfd0da0b0257d73b8e5dd2c8e9955bd72eb1f"
Oct 31 01:13:22.636062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.636037    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:22.636109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.636097    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:22.636147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.636126    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:22.645044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.645026    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:22.662720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.662696    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:22.789852 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.789151    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:22.957989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.954850    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"]
Oct 31 01:13:22.998381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.997749    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"]
Oct 31 01:13:23.002049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.999675    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.002049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:22.999890    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.002049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.000074    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.025108 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.024465    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.025108 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.024557    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.025108 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.024618    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.136319 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.134351    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"]
Oct 31 01:13:23.154714 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:23.154640    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcae1e4_b7a3_4677_807e_5a8a87bbe612.slice/crio-60cb5c110dffdf71c96fad1b2792462f906057be61ae76c8e989fbe81f741e71 WatchSource:0}: Error finding container 60cb5c110dffdf71c96fad1b2792462f906057be61ae76c8e989fbe81f741e71: Status 404 returned error can't find the container with id 60cb5c110dffdf71c96fad1b2792462f906057be61ae76c8e989fbe81f741e71
Oct 31 01:13:23.175945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.173407    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.176211 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.176178    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.176317 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.176249    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.215189 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:23.215131    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9658b33_0dd9_42b5_9e2f_4e406b19b988.slice/crio-bab41ca513891a6cbc0c49606713270bad1b20f84d9fbda052f0077d7fef3913 WatchSource:0}: Error finding container bab41ca513891a6cbc0c49606713270bad1b20f84d9fbda052f0077d7fef3913: Status 404 returned error can't find the container with id bab41ca513891a6cbc0c49606713270bad1b20f84d9fbda052f0077d7fef3913
Oct 31 01:13:23.248686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.248282    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"]
Oct 31 01:13:23.272248 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.272209    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.274535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.274504    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.274929 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.274908    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:13:23.866627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.860059    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" event={"ID":"21bfd530-3cf3-4784-81f1-d78f9fb72192","Type":"ContainerStarted","Data":"2701c160c1b02c9a3b1ead6165e7f18a40200af45c2bc316b8b020da3ceec1b7"}
Oct 31 01:13:23.866627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.860230    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" event={"ID":"21bfd530-3cf3-4784-81f1-d78f9fb72192","Type":"ContainerStarted","Data":"2e38adcf870cec2af1527e2b6b08759c510aa97201f0be903628aab258426943"}
Oct 31 01:13:23.866880 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.866842    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" event={"ID":"e9658b33-0dd9-42b5-9e2f-4e406b19b988","Type":"ContainerStarted","Data":"d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969"}
Oct 31 01:13:23.866941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.866897    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" event={"ID":"e9658b33-0dd9-42b5-9e2f-4e406b19b988","Type":"ContainerStarted","Data":"bab41ca513891a6cbc0c49606713270bad1b20f84d9fbda052f0077d7fef3913"}
Oct 31 01:13:23.871622 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.871578    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" event={"ID":"2fcae1e4-b7a3-4677-807e-5a8a87bbe612","Type":"ContainerStarted","Data":"9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053"}
Oct 31 01:13:23.871818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.871803    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" event={"ID":"2fcae1e4-b7a3-4677-807e-5a8a87bbe612","Type":"ContainerStarted","Data":"60cb5c110dffdf71c96fad1b2792462f906057be61ae76c8e989fbe81f741e71"}
Oct 31 01:13:23.874546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.874523    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" event={"ID":"328e2256-e817-4994-90c8-4c27a7c1f5ab","Type":"ContainerStarted","Data":"9a887b9d2170ef5204f0c57358f1bc1ddab4e6d1562ef268cbd2c1269ba7295c"}
Oct 31 01:13:23.874744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.874730    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" event={"ID":"328e2256-e817-4994-90c8-4c27a7c1f5ab","Type":"ContainerStarted","Data":"84c66f4f4b2660c8038052d7a9ba9cc4ba442cd1651e1901f9f55bb27ed13041"}
Oct 31 01:13:23.889624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.886964    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" event={"ID":"87c94909-be59-4713-8244-3444fe3c8b0a","Type":"ContainerStarted","Data":"b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce"}
Oct 31 01:13:23.889624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.887024    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" event={"ID":"87c94909-be59-4713-8244-3444fe3c8b0a","Type":"ContainerStarted","Data":"e7b4b936c2e0e4e9a9e395a5bab41ef2e395650c6ab78c3196e133421c35b853"}
Oct 31 01:13:23.970628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:23.970264    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" event={"ID":"5cb2044d-95c3-409b-862d-d4758727f4bb","Type":"ContainerStarted","Data":"767f37acc4134f0758a740ea3ec64a4d2dc223f8c4984af5525b8045882fdf06"}
Oct 31 01:13:24.031114 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.031055    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podStartSLOduration=2.031034745 podStartE2EDuration="2.031034745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:23.952059016 +0000 UTC m=+1000.939170993" watchObservedRunningTime="2026-10-31 01:13:24.031034745 +0000 UTC m=+1001.018146723"
Oct 31 01:13:24.055961 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.048332    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podStartSLOduration=2.048308063 podStartE2EDuration="2.048308063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:23.911452941 +0000 UTC m=+1000.898564919" watchObservedRunningTime="2026-10-31 01:13:24.048308063 +0000 UTC m=+1001.035420041"
Oct 31 01:13:24.279202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.279068    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:24.279495 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.279440    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podStartSLOduration=2.27942442 podStartE2EDuration="2.27942442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:23.978374869 +0000 UTC m=+1000.965486848" watchObservedRunningTime="2026-10-31 01:13:24.27942442 +0000 UTC m=+1001.266536397"
Oct 31 01:13:24.287029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.283736    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:24.287029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.283791    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:24.307832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.307800    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:24.318767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.318732    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:24.319034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.319005    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:24.336928 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.336861    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podStartSLOduration=2.3368438129999998 podStartE2EDuration="2.336843813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:24.028146688 +0000 UTC m=+1001.015258666" watchObservedRunningTime="2026-10-31 01:13:24.336843813 +0000 UTC m=+1001.323955790"
Oct 31 01:13:24.421626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.417790    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podStartSLOduration=3.417766269 podStartE2EDuration="3.417766269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:24.121760543 +0000 UTC m=+1001.108872500" watchObservedRunningTime="2026-10-31 01:13:24.417766269 +0000 UTC m=+1001.404878246"
Oct 31 01:13:24.536686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.530384    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podStartSLOduration=3.530362185 podStartE2EDuration="3.530362185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:24.181136855 +0000 UTC m=+1001.168248832" watchObservedRunningTime="2026-10-31 01:13:24.530362185 +0000 UTC m=+1001.517474162"
Oct 31 01:13:24.662434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.662396    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:24.664224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.664199    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:24.669829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.669795    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:24.669829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.671051    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:24.669829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.669865    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:24.669829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.669907    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:24.669829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.671354    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:24.681785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.681752    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:24.682645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.682585    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:24.799839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.796713    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:24.799839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.796790    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:24.799839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:24.797141    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:25.090813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.090702    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:25.134641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.122632    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:25.288103 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.288063    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:25.288370 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.288343    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:25.312091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.312056    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:25.312381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.312353    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:25.380756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.380703    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:25.453769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.453722    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:25.552819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.552757    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:25.637686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.615094    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:25.712628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.640199    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:25.712628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.680359    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:25.712628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.709240    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:25.712628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.680414    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:25.712628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.709295    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:25.737631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.709202    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:25.800618 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.800557    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:25.806704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.806667    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:25.806964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:25.800861    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:26.097329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.094701    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:26.097329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.095199    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:13:26.195629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.167548    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:26.346686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.289088    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:26.347003 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.346969    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:26.347194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.296081    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:26.347389 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.347322    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:26.348042 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.348008    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:13:26.429143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.407411    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:26.455860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.455833    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:13:26.509626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.508614    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:26.509626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.509069    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:13:26.570630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.557119    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:26.570630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.557595    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:13:26.751636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.725776    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:26.878554 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.849005    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:26.880720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.880546    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:13:26.881045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.881022    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:13:26.948289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.948250    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:26.948550 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.948522    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:26.948792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.948774    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:26.948904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.948887    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:26.949047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.949034    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:26.949135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:26.949121    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:27.130641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.126760    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:27.130641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.126823    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:27.149424 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.149374    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:27.235790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.235738    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:27.273508 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.273471    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:27.273767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.273735    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:27.318211 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.318176    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:27.318474 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.318445    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:27.342670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.338400    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:27.520776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.466850    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:27.619628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.618097    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:27.687630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.629522    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:27.809404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.733201    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:27.810061 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.810026    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:27.819626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.733242    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:27.841632 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.832765    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:27.841632 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.733280    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:27.926628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.880738    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:27.973584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.973527    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:27.984512 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.984474    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:28.115633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:27.832685    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.115633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.114252    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:28.169341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.169299    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:28.270943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.270826    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.270943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.270882    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:28.365738 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.365641    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:28.413046 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.404661    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.443557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.443515    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:28.443951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.426986    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" podUID="eeda0349-fac8-42fc-a73d-f81c06888419" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.24:6080/livez\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:28.605630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.569837    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:28.620364 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.619830    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:28.841630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.775854    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.842659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.842617    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:28.843188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.840878    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.843342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.843324    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:28.844101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.844073    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:28.892689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.874257    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.892689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.874420    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:28.929629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.929256    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:28.929629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:28.929317    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:29.008170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.008127    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:29.047662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.034674    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:29.150390 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.150286    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" podUID="eeda0349-fac8-42fc-a73d-f81c06888419" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.24:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:29.189897 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.189861    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:29.289664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.284548    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:29.289664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.284624    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:29.304764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.304733    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:29.312830 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.312794    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:29.416748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.415044    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:29.436061 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.435738    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:29.642791 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.642736    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:29.645565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.645533    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:29.771687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.666445    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:29.771687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.746498    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:29.771687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.687344    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:29.771687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.746567    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:29.771687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.730036    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:29.771687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.769680    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:29.854634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.836711    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:29.885639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:29.872718    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:30.162498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.041728    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.162498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.041857    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:30.162498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.048837    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:30.162498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.049232    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:30.283643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.190737    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:30.325816 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.325762    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.374649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.373982    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:30.374649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.326290    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.374649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.374117    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:30.465688 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.463385    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:30.648048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.647991    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:30.657276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.657240    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.657578 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.657548    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:30.668232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.657967    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.668502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.668458    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:30.694719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.694678    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.694905 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.694741    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:30.780699 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.765167    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:30.810655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.798884    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:30.810655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.798945    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:30.879263 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:30.879225    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:31.054687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.053786    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:31.077685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.055123    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:31.218629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.217908    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:31.272627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.272040    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:31.272627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.272105    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:31.297987 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.297949    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:31.298535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.298503    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:31.479629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.479441    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:13:31.752684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.682001    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:31.752684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.682064    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:31.786118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.782725    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:31.786118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.782926    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:31.786118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.785258    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:31.786118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.785337    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:31.786118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.785398    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:31.786118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.785419    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:31.922178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.791159    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:31.922178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.901546    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:31.922178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:31.902744    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:32.360779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.218677    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:32.360779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.223789    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Oct 31 01:13:32.360779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.225713    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:32.360779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.267104    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:32.360779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.311250    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:32.360779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.311320    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:32.409190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.409146    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:32.472562 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.472502    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:32.481348 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.481269    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:32.643942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.643844    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:32.644201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.644170    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:32.647493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.647465    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:32.651699 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.651663    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:32.666255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.666227    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:32.666496 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.666472    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:32.784802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.784746    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:32.787721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.787673    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:32.790996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.790967    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:32.791230 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.791204    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:32.911689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.909327    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:32.912119 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:32.912079    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:33.222828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.222704    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:33.247342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.246857    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:33.287839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.287779    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:33.311492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.310954    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:33.311492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.311011    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:33.311492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.311397    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:33.311492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.311429    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:33.484900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.484783    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:33.634140 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.634104    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:33.634426 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.634383    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:33.647416 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.647381    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:33.647718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.647668    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:33.671370 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.671321    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:33.671743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.671713    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:33.761644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.761536    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:13:33.786121 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.786076    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:34.007790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.788102    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:34.009019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.797001    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.015775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.015686    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:34.015955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:33.959719    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:34.157625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.153694    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:34.247784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.247738    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:34.248415 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.248384    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:34.271680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.268969    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.271680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.269026    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 01:13:34.343629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.290709    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:34.343629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.324770    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.343629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.324830    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 31 01:13:34.613771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.613725    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": context deadline exceeded"
Oct 31 01:13:34.636093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.636049    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.636364 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.636338    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:34.662163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.662128    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.662429 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.662405    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:34.700258 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.700225    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.700517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.700489    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:34.796625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.796203    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:34.796625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:34.796259    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:35.007842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.007741    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:35.050636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.011800    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:35.050812 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.033727    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:35.410624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.402781    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:35.410624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.404986    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:35.480526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.468335    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:35.677397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.677266    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:35.880566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.880531    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 31 01:13:35.880919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:35.880875    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 31 01:13:36.109886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.109826    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:36.110170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.110126    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:36.132780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.111998    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:36.133094 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.112500    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:36.158016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.112553    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 31 01:13:36.216637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.209586    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 31 01:13:36.216637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.175307    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:36.216637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.179982    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:13:36.216637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.209788    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:13:36.216637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.159771    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:36.497058 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.496790    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:36.560030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.549943    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:36.591624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.560240    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:36.595620 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.592990    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:13:36.613841 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.607540    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:36.649625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.632665    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:36.649625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.644967    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:36.649625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.645981    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:36.665146 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.665121    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:13:36.665732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.665690    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 31 01:13:36.689664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.689624    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 31 01:13:36.726113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.716110    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:36.786783 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.786703    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:13:36.820540 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.820510    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:13:36.822646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.822584    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 01:13:36.822827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:36.822801    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 01:13:37.226783 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.226722    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:37.280674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.280640    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:13:37.384546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.378990    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:37.386137 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.385982    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:37.398379 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.395097    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:13:37.519681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.501177    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:37.612506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.612477    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:37.614744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.614712    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:37.615303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.615265    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:13:37.619534 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.619515    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:13:37.676772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.676742    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:37.790246 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.790169    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:37.860259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.860215    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:37.895959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:37.895909    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:38.350873 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:38.350557    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:13:38.363251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:38.362897    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:13:39.176684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:39.176639    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:39.426628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:39.421765    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:39.691779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:39.691735    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:39.821779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:39.821734    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:40.185820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.185778    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:40.187079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.187052    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:13:40.446890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.446360    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:40.449741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.449709    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:13:40.703917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.701737    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:40.703917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.702560    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:13:40.748802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.748749    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:40.829514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.829466    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:40.844757 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:40.830488    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:13:41.134693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.054312    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:41.185630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.076701    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:41.210629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.187159    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:41.461067 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.455647    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:41.733706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.714218    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:41.751747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.751705    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:41.753057 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.752631    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:13:41.835627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:41.831244    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:42.120813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.120768    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:42.250628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.210023    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:42.250628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.210659    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:42.250628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.211021    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:13:42.250628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.211080    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:13:42.250628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.221711    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:42.509691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.496748    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:42.744886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.720512    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:42.771817 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:42.771709    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:43.031692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.031157    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:43.031692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.031251    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:43.051633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.033465    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:43.051633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.034647    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/health\": context deadline exceeded"
Oct 31 01:13:43.214134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.214092    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:43.214761 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.214729    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:43.527631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.526763    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:43.768946 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:43.768898    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:44.084629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.037694    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:44.084629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.070036    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:44.137093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.137042    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:44.140791 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.140746    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:44.151629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.139525    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:44.196793 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.196748    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": context deadline exceeded"
Oct 31 01:13:44.216804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.216759    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:44.217406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.217373    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:44.291630 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:13:44.288528    8029 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.001s"
Oct 31 01:13:44.383689 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.383648    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"]
Oct 31 01:13:44.490963 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.490930    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.494822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.491557    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"]
Oct 31 01:13:44.513109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.513072    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:13:44.514005 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.513408    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:13:44.514156 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.513499    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:13:44.514245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.513655    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:13:44.530745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.530707    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:44.590593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.590556    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9gh5\" (UniqueName: \"kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.593589 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.593562    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.598178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.598149    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.718690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.715821    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:44.721839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.721806    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9gh5\" (UniqueName: \"kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.722066 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.722047    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.722775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.722746    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.772240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.762646    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9gh5\" (UniqueName: \"kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.772240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.769999    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.772240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.770083    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:44.787629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.772741    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:44.794299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:44.794271    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:45.165631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.164705    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:13:45.169637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.166101    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:45.169637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.166407    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:45.216278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.215942    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:45.216278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.216001    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:45.230239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.227810    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:45.230239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.228257    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:13:45.367880 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.367841    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:45.368229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.368196    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:45.404550 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.404185    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:45.404550 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.404263    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:45.552302 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.552182    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:45.772627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:45.770801    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:13:46.022875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.022838    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.057640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.056531    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.057640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.056613    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.058715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.023174    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.156793 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.156757    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.177164    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.178274    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.178978    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.233152    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.233206    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.266436    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.247370    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.273710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.266508    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.471628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.451480    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.471628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.451560    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.471628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.470757    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.471628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.470819    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.529978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.515673    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:13:46.529978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.515791    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.558561 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.557238    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.763629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.763515    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"]
Oct 31 01:13:46.779333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:46.775707    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:46.898797 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:46.869506    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4898e302_d834_49c5_ada3_f507d6dd6889.slice/crio-339de0c2c241775a27ed07d737a931bee9a3c01d2ab1281650265ad8d39f0652 WatchSource:0}: Error finding container 339de0c2c241775a27ed07d737a931bee9a3c01d2ab1281650265ad8d39f0652: Status 404 returned error can't find the container with id 339de0c2c241775a27ed07d737a931bee9a3c01d2ab1281650265ad8d39f0652
Oct 31 01:13:47.241092 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.241024    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:47.274657 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.274582    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:47.312720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.277680    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:47.314237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.277744    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:47.360691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.351671    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:47.431910 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.428072    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:13:48.129697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.619152    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:48.129697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:47.878977    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:48.332331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.332291    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:13:48.332990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.332930    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:13:48.333459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.333428    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:48.457905 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.379859    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:48.465974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.465476    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:13:48.509553 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:13:48.509519    8029 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.15s"
Oct 31 01:13:48.509937 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.509913    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp" event={"ID":"4898e302-d834-49c5-ada3-f507d6dd6889","Type":"ContainerStarted","Data":"339de0c2c241775a27ed07d737a931bee9a3c01d2ab1281650265ad8d39f0652"}
Oct 31 01:13:48.519556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.518974    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:13:48.521733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.519740    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:13:48.521733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.520786    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:13:48.526858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.524993    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:13:48.526858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.526077    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:13:48.527342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.527320    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:13:48.529059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:48.529037    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:13:49.400138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.400101    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"]
Oct 31 01:13:49.407136 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.407105    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.409079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.409036    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"]
Oct 31 01:13:49.457040 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:13:49.456999    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:13:49.465198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.465145    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.465359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.465307    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4w2z\" (UniqueName: \"kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.465498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.465474    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.495298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.495263    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"]
Oct 31 01:13:49.501578 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.501543    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.527373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.526168    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"]
Oct 31 01:13:49.535741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.535704    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp" event={"ID":"4898e302-d834-49c5-ada3-f507d6dd6889","Type":"ContainerStarted","Data":"342d6f1e05db119c536e2c0bf250f2d53b30304c178960224fb857d9353947ce"}
Oct 31 01:13:49.556438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.556387    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp" podStartSLOduration=7.556372045 podStartE2EDuration="7.556372045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:49.555866681 +0000 UTC m=+1026.542978658" watchObservedRunningTime="2026-10-31 01:13:49.556372045 +0000 UTC m=+1026.543484021"
Oct 31 01:13:49.566932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.566903    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.567079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.567063    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb4ct\" (UniqueName: \"kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.567199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.567176    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.567272 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.567214    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.567374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.567360    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.567441 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.567382    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4w2z\" (UniqueName: \"kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.567736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.567713    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.569815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.569798    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.575770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.575753    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4w2z\" (UniqueName: \"kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z\") pod \"maas-api-revoke-sub-c46d0f47-hsxhw\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.648651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.648614    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"]
Oct 31 01:13:49.653752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.653652    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.671749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.671660    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"]
Oct 31 01:13:49.707886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.703208    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.707886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.703669    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb4ct\" (UniqueName: \"kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.707886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.703785    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.707886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.707407    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.707886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.707761    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.712361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.712338    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb4ct\" (UniqueName: \"kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct\") pod \"maas-api-revoke-sub-9d0fa3f3-kcxjf\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.732398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.732366    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:49.805749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.805714    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqbs7\" (UniqueName: \"kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.806998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.806078    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.806998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.806126    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.836113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.835846    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:49.919900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.919802    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqbs7\" (UniqueName: \"kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.920205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.920181    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.920313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.920244    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.928163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.921970    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.928163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.923878    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:49.938849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:49.938830    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqbs7\" (UniqueName: \"kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7\") pod \"maas-api-revoke-sub-8c26e61b-2h2tj\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:50.027999 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:50.027954    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0ca893_722d_4c70_8616_86035ddfbc33.slice/crio-b2892230bd0f9171fe722293c5b12d221b10338f2a5f5b627cfa693060f51cd6 WatchSource:0}: Error finding container b2892230bd0f9171fe722293c5b12d221b10338f2a5f5b627cfa693060f51cd6: Status 404 returned error can't find the container with id b2892230bd0f9171fe722293c5b12d221b10338f2a5f5b627cfa693060f51cd6
Oct 31 01:13:50.028887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.028859    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"]
Oct 31 01:13:50.076271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.076246    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:50.163652 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:50.163615    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d055e5_9b62_404b_8c99_46f4c0ac91d9.slice/crio-2099263ee3a5aa1b8ede2c7bbb07b4c1a3f68a15d9da1825d1b96d971802e824 WatchSource:0}: Error finding container 2099263ee3a5aa1b8ede2c7bbb07b4c1a3f68a15d9da1825d1b96d971802e824: Status 404 returned error can't find the container with id 2099263ee3a5aa1b8ede2c7bbb07b4c1a3f68a15d9da1825d1b96d971802e824
Oct 31 01:13:50.167934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.167904    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"]
Oct 31 01:13:50.366704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.366676    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"]
Oct 31 01:13:50.379920 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:50.379875    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f56b97_0180_4da3_b24b_6b4cc112e60c.slice/crio-096d7447fc044c86e84da184b15c997692f934db3b7939a3a673b90a720beecc WatchSource:0}: Error finding container 096d7447fc044c86e84da184b15c997692f934db3b7939a3a673b90a720beecc: Status 404 returned error can't find the container with id 096d7447fc044c86e84da184b15c997692f934db3b7939a3a673b90a720beecc
Oct 31 01:13:50.558117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.558087    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf" event={"ID":"d9d055e5-9b62-404b-8c99-46f4c0ac91d9","Type":"ContainerStarted","Data":"2099263ee3a5aa1b8ede2c7bbb07b4c1a3f68a15d9da1825d1b96d971802e824"}
Oct 31 01:13:50.559583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.559555    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj" event={"ID":"38f56b97-0180-4da3-b24b-6b4cc112e60c","Type":"ContainerStarted","Data":"096d7447fc044c86e84da184b15c997692f934db3b7939a3a673b90a720beecc"}
Oct 31 01:13:50.561331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.561309    8029 generic.go:356] "Generic (PLEG): container finished" podID="4898e302-d834-49c5-ada3-f507d6dd6889" containerID="342d6f1e05db119c536e2c0bf250f2d53b30304c178960224fb857d9353947ce" exitCode=0
Oct 31 01:13:50.561417 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.561391    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp" event={"ID":"4898e302-d834-49c5-ada3-f507d6dd6889","Type":"ContainerDied","Data":"342d6f1e05db119c536e2c0bf250f2d53b30304c178960224fb857d9353947ce"}
Oct 31 01:13:50.562990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.562971    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw" event={"ID":"ac0ca893-722d-4c70-8616-86035ddfbc33","Type":"ContainerStarted","Data":"888eedd6ef34d09646982fd0f582b356bade2e93835fe3061bb4091454e016e2"}
Oct 31 01:13:50.563070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.563005    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw" event={"ID":"ac0ca893-722d-4c70-8616-86035ddfbc33","Type":"ContainerStarted","Data":"b2892230bd0f9171fe722293c5b12d221b10338f2a5f5b627cfa693060f51cd6"}
Oct 31 01:13:50.605809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:50.605745    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw" podStartSLOduration=1.6057218930000001 podStartE2EDuration="1.605721893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:50.600192433 +0000 UTC m=+1027.587304411" watchObservedRunningTime="2026-10-31 01:13:50.605721893 +0000 UTC m=+1027.592833870"
Oct 31 01:13:51.574895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.574861    8029 generic.go:356] "Generic (PLEG): container finished" podID="ac0ca893-722d-4c70-8616-86035ddfbc33" containerID="888eedd6ef34d09646982fd0f582b356bade2e93835fe3061bb4091454e016e2" exitCode=0
Oct 31 01:13:51.575343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.574938    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw" event={"ID":"ac0ca893-722d-4c70-8616-86035ddfbc33","Type":"ContainerDied","Data":"888eedd6ef34d09646982fd0f582b356bade2e93835fe3061bb4091454e016e2"}
Oct 31 01:13:51.576758 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.576733    8029 generic.go:356] "Generic (PLEG): container finished" podID="d9d055e5-9b62-404b-8c99-46f4c0ac91d9" containerID="96907b566a5a1c3e13a27c8849681c1eacf1f389a2741a76a3424f2d753882c2" exitCode=0
Oct 31 01:13:51.576898 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.576812    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf" event={"ID":"d9d055e5-9b62-404b-8c99-46f4c0ac91d9","Type":"ContainerDied","Data":"96907b566a5a1c3e13a27c8849681c1eacf1f389a2741a76a3424f2d753882c2"}
Oct 31 01:13:51.578446 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.578423    8029 generic.go:356] "Generic (PLEG): container finished" podID="38f56b97-0180-4da3-b24b-6b4cc112e60c" containerID="bb014f206d2a2f852b83f397497405222ac9315b5f2c1668130ed45862fe30c6" exitCode=0
Oct 31 01:13:51.578553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.578502    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj" event={"ID":"38f56b97-0180-4da3-b24b-6b4cc112e60c","Type":"ContainerDied","Data":"bb014f206d2a2f852b83f397497405222ac9315b5f2c1668130ed45862fe30c6"}
Oct 31 01:13:51.755570 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.755503    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:51.873013 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.872981    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca\") pod \"4898e302-d834-49c5-ada3-f507d6dd6889\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") "
Oct 31 01:13:51.873171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.873041    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5\" (UniqueName: \"kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5\") pod \"4898e302-d834-49c5-ada3-f507d6dd6889\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") "
Oct 31 01:13:51.873171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.873162    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca\") pod \"4898e302-d834-49c5-ada3-f507d6dd6889\" (UID: \"4898e302-d834-49c5-ada3-f507d6dd6889\") "
Oct 31 01:13:51.873361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.873336    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca" pod "4898e302-d834-49c5-ada3-f507d6dd6889" (UID: "4898e302-d834-49c5-ada3-f507d6dd6889"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:51.873778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.873761    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:51.876168 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.876146    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5" pod "4898e302-d834-49c5-ada3-f507d6dd6889" (UID: "4898e302-d834-49c5-ada3-f507d6dd6889"). InnerVolumeSpecName "kube-api-access-d9gh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:51.876336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.876315    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca" pod "4898e302-d834-49c5-ada3-f507d6dd6889" (UID: "4898e302-d834-49c5-ada3-f507d6dd6889"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:51.977189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.977148    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9gh5\" (UniqueName: \"kubernetes.io/projected/4898e302-d834-49c5-ada3-f507d6dd6889-kube-api-access-d9gh5\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:51.977189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:51.977192    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4898e302-d834-49c5-ada3-f507d6dd6889-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:52.590721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.590692    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"
Oct 31 01:13:52.591196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.590737    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp" event={"ID":"4898e302-d834-49c5-ada3-f507d6dd6889","Type":"ContainerDied","Data":"339de0c2c241775a27ed07d737a931bee9a3c01d2ab1281650265ad8d39f0652"}
Oct 31 01:13:52.591196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.590791    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="339de0c2c241775a27ed07d737a931bee9a3c01d2ab1281650265ad8d39f0652"
Oct 31 01:13:52.882749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.882029    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:52.894998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.892958    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct\" (UniqueName: \"kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct\") pod \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") "
Oct 31 01:13:52.894998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.893596    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca\") pod \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") "
Oct 31 01:13:52.894998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.894231    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca" pod "d9d055e5-9b62-404b-8c99-46f4c0ac91d9" (UID: "d9d055e5-9b62-404b-8c99-46f4c0ac91d9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:52.898497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.898257    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca\") pod \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\" (UID: \"d9d055e5-9b62-404b-8c99-46f4c0ac91d9\") "
Oct 31 01:13:52.901012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.900971    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct" pod "d9d055e5-9b62-404b-8c99-46f4c0ac91d9" (UID: "d9d055e5-9b62-404b-8c99-46f4c0ac91d9"). InnerVolumeSpecName "kube-api-access-wb4ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:52.905493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.905469    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:52.905852 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.905833    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb4ct\" (UniqueName: \"kubernetes.io/projected/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-kube-api-access-wb4ct\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:52.906855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.906831    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca" pod "d9d055e5-9b62-404b-8c99-46f4c0ac91d9" (UID: "d9d055e5-9b62-404b-8c99-46f4c0ac91d9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:52.968482 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.968134    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:52.975144 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:52.975121    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.015045    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca\") pod \"ac0ca893-722d-4c70-8616-86035ddfbc33\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") "
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.015232    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca\") pod \"38f56b97-0180-4da3-b24b-6b4cc112e60c\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") "
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.015287    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z\" (UniqueName: \"kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z\") pod \"ac0ca893-722d-4c70-8616-86035ddfbc33\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") "
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.015356    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7\" (UniqueName: \"kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7\") pod \"38f56b97-0180-4da3-b24b-6b4cc112e60c\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") "
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.015406    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca\") pod \"ac0ca893-722d-4c70-8616-86035ddfbc33\" (UID: \"ac0ca893-722d-4c70-8616-86035ddfbc33\") "
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.015490    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca\") pod \"38f56b97-0180-4da3-b24b-6b4cc112e60c\" (UID: \"38f56b97-0180-4da3-b24b-6b4cc112e60c\") "
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.016425    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d9d055e5-9b62-404b-8c99-46f4c0ac91d9-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.022649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.016798    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca" pod "38f56b97-0180-4da3-b24b-6b4cc112e60c" (UID: "38f56b97-0180-4da3-b24b-6b4cc112e60c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:53.031218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.031177    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca" pod "ac0ca893-722d-4c70-8616-86035ddfbc33" (UID: "ac0ca893-722d-4c70-8616-86035ddfbc33"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:53.031695 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.031668    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca" pod "ac0ca893-722d-4c70-8616-86035ddfbc33" (UID: "ac0ca893-722d-4c70-8616-86035ddfbc33"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:53.031995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.031969    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z" pod "ac0ca893-722d-4c70-8616-86035ddfbc33" (UID: "ac0ca893-722d-4c70-8616-86035ddfbc33"). InnerVolumeSpecName "kube-api-access-k4w2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:53.033002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.032971    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7" pod "38f56b97-0180-4da3-b24b-6b4cc112e60c" (UID: "38f56b97-0180-4da3-b24b-6b4cc112e60c"). InnerVolumeSpecName "kube-api-access-dqbs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:53.034158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.034132    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca" pod "38f56b97-0180-4da3-b24b-6b4cc112e60c" (UID: "38f56b97-0180-4da3-b24b-6b4cc112e60c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:53.157954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.130066    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4w2z\" (UniqueName: \"kubernetes.io/projected/ac0ca893-722d-4c70-8616-86035ddfbc33-kube-api-access-k4w2z\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.157954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.130109    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqbs7\" (UniqueName: \"kubernetes.io/projected/38f56b97-0180-4da3-b24b-6b4cc112e60c-kube-api-access-dqbs7\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.157954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.130125    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.157954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.130140    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.157954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.130155    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0ca893-722d-4c70-8616-86035ddfbc33-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.157954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.130170    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/38f56b97-0180-4da3-b24b-6b4cc112e60c-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:53.608375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.608339    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw" event={"ID":"ac0ca893-722d-4c70-8616-86035ddfbc33","Type":"ContainerDied","Data":"b2892230bd0f9171fe722293c5b12d221b10338f2a5f5b627cfa693060f51cd6"}
Oct 31 01:13:53.608997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.608960    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2892230bd0f9171fe722293c5b12d221b10338f2a5f5b627cfa693060f51cd6"
Oct 31 01:13:53.609138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.609070    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"
Oct 31 01:13:53.613483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.613452    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf" event={"ID":"d9d055e5-9b62-404b-8c99-46f4c0ac91d9","Type":"ContainerDied","Data":"2099263ee3a5aa1b8ede2c7bbb07b4c1a3f68a15d9da1825d1b96d971802e824"}
Oct 31 01:13:53.613695 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.613617    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2099263ee3a5aa1b8ede2c7bbb07b4c1a3f68a15d9da1825d1b96d971802e824"
Oct 31 01:13:53.613777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.613762    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"
Oct 31 01:13:53.617109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.617086    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"
Oct 31 01:13:53.617228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.617146    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj" event={"ID":"38f56b97-0180-4da3-b24b-6b4cc112e60c","Type":"ContainerDied","Data":"096d7447fc044c86e84da184b15c997692f934db3b7939a3a673b90a720beecc"}
Oct 31 01:13:53.617228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.617209    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="096d7447fc044c86e84da184b15c997692f934db3b7939a3a673b90a720beecc"
Oct 31 01:13:53.900455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.900343    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"]
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901805    8029 cpu_manager.go:414] "containerMap: removing container" podUID="4898e302-d834-49c5-ada3-f507d6dd6889" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901833    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4898e302-d834-49c5-ada3-f507d6dd6889" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901861    8029 cpu_manager.go:414] "containerMap: removing container" podUID="38f56b97-0180-4da3-b24b-6b4cc112e60c" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901869    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38f56b97-0180-4da3-b24b-6b4cc112e60c" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901891    8029 cpu_manager.go:414] "containerMap: removing container" podUID="ac0ca893-722d-4c70-8616-86035ddfbc33" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901898    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac0ca893-722d-4c70-8616-86035ddfbc33" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901912    8029 cpu_manager.go:414] "containerMap: removing container" podUID="d9d055e5-9b62-404b-8c99-46f4c0ac91d9" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.901919    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9d055e5-9b62-404b-8c99-46f4c0ac91d9" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.902115    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac0ca893-722d-4c70-8616-86035ddfbc33" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.902129    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9d055e5-9b62-404b-8c99-46f4c0ac91d9" containerName="revoke-keys"
Oct 31 01:13:53.902141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.902152    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="4898e302-d834-49c5-ada3-f507d6dd6889" containerName="revoke-keys"
Oct 31 01:13:53.902839 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.902163    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="38f56b97-0180-4da3-b24b-6b4cc112e60c" containerName="revoke-keys"
Oct 31 01:13:53.931020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.930916    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"]
Oct 31 01:13:53.931200 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.931030    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:53.940070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.936307    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:13:53.940070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.936980    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:13:53.940070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.938146    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:13:53.940070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:53.938421    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:13:54.060437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.060404    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"]
Oct 31 01:13:54.064978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.064958    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.065742 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.065718    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.065871 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.065847    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.066049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.066033    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbz7v\" (UniqueName: \"kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.073861 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.073817    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"]
Oct 31 01:13:54.167511 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.167416    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbz7v\" (UniqueName: \"kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.167705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.167572    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.167782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.167755    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.167859 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.167843    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.167939 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.167927    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqj2v\" (UniqueName: \"kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.168034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.167992    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.168422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.168392    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.170531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.170508    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.175825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.175802    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbz7v\" (UniqueName: \"kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v\") pod \"maas-api-revoke-sub-6d54aef9-97tvp\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.269744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.269708    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqj2v\" (UniqueName: \"kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.269927 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.269762    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.269927 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.269919    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.270541 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.270521    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.271384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.271363    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:54.272382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.272361    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.277470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.277443    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqj2v\" (UniqueName: \"kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v\") pod \"maas-api-revoke-sub-5a5d2acf-z87qc\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.385438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.385402    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:54.573455 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:54.573425    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88ecd35_15ca_4dab_89e3_d3d02f4264a9.slice/crio-e12a8701305e8703868fd8dab13c4be5e010c4aab6557e8c65a99ded33532373 WatchSource:0}: Error finding container e12a8701305e8703868fd8dab13c4be5e010c4aab6557e8c65a99ded33532373: Status 404 returned error can't find the container with id e12a8701305e8703868fd8dab13c4be5e010c4aab6557e8c65a99ded33532373
Oct 31 01:13:54.575117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.575092    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"]
Oct 31 01:13:54.627559 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.627531    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp" event={"ID":"e88ecd35-15ca-4dab-89e3-d3d02f4264a9","Type":"ContainerStarted","Data":"e12a8701305e8703868fd8dab13c4be5e010c4aab6557e8c65a99ded33532373"}
Oct 31 01:13:54.687969 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:54.687944    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47907ba_7a0c_4fd1_860c_7d1bccde84be.slice/crio-de9e927c7280737ea3c8baab978288220708abaf7c5dfc58ca9df3fe55666698 WatchSource:0}: Error finding container de9e927c7280737ea3c8baab978288220708abaf7c5dfc58ca9df3fe55666698: Status 404 returned error can't find the container with id de9e927c7280737ea3c8baab978288220708abaf7c5dfc58ca9df3fe55666698
Oct 31 01:13:54.688651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:54.688596    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"]
Oct 31 01:13:55.639840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:55.639812    8029 generic.go:356] "Generic (PLEG): container finished" podID="e88ecd35-15ca-4dab-89e3-d3d02f4264a9" containerID="f0dfc0b6f73ec515eea6bfda0e290b20359a82964b5056517c07b4d48b8998bf" exitCode=0
Oct 31 01:13:55.640235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:55.639892    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp" event={"ID":"e88ecd35-15ca-4dab-89e3-d3d02f4264a9","Type":"ContainerDied","Data":"f0dfc0b6f73ec515eea6bfda0e290b20359a82964b5056517c07b4d48b8998bf"}
Oct 31 01:13:55.641513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:55.641493    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc" event={"ID":"f47907ba-7a0c-4fd1-860c-7d1bccde84be","Type":"ContainerStarted","Data":"3ac34bd6b7940bce6155838da06254699d5b4b116656b923892371ba25e28ca1"}
Oct 31 01:13:55.641646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:55.641530    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc" event={"ID":"f47907ba-7a0c-4fd1-860c-7d1bccde84be","Type":"ContainerStarted","Data":"de9e927c7280737ea3c8baab978288220708abaf7c5dfc58ca9df3fe55666698"}
Oct 31 01:13:55.672961 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:55.672916    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc" podStartSLOduration=1.672905174 podStartE2EDuration="1.672905174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:55.66979795 +0000 UTC m=+1032.656909939" watchObservedRunningTime="2026-10-31 01:13:55.672905174 +0000 UTC m=+1032.660017149"
Oct 31 01:13:56.659919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:56.659880    8029 generic.go:356] "Generic (PLEG): container finished" podID="f47907ba-7a0c-4fd1-860c-7d1bccde84be" containerID="3ac34bd6b7940bce6155838da06254699d5b4b116656b923892371ba25e28ca1" exitCode=0
Oct 31 01:13:56.660530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:56.660022    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc" event={"ID":"f47907ba-7a0c-4fd1-860c-7d1bccde84be","Type":"ContainerDied","Data":"3ac34bd6b7940bce6155838da06254699d5b4b116656b923892371ba25e28ca1"}
Oct 31 01:13:56.931347 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:56.931268    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:57.066505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.066475    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca\") pod \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") "
Oct 31 01:13:57.066718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.066647    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca\") pod \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") "
Oct 31 01:13:57.066801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.066721    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v\" (UniqueName: \"kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v\") pod \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\" (UID: \"e88ecd35-15ca-4dab-89e3-d3d02f4264a9\") "
Oct 31 01:13:57.066940 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.066914    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca" pod "e88ecd35-15ca-4dab-89e3-d3d02f4264a9" (UID: "e88ecd35-15ca-4dab-89e3-d3d02f4264a9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:57.067354 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.067333    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:57.068907 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.068881    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca" pod "e88ecd35-15ca-4dab-89e3-d3d02f4264a9" (UID: "e88ecd35-15ca-4dab-89e3-d3d02f4264a9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:57.069008 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.068983    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v" pod "e88ecd35-15ca-4dab-89e3-d3d02f4264a9" (UID: "e88ecd35-15ca-4dab-89e3-d3d02f4264a9"). InnerVolumeSpecName "kube-api-access-pbz7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:57.168567 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.168542    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:57.168567 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.168567    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbz7v\" (UniqueName: \"kubernetes.io/projected/e88ecd35-15ca-4dab-89e3-d3d02f4264a9-kube-api-access-pbz7v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:57.374638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.374583    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"]
Oct 31 01:13:57.375288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.375275    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e88ecd35-15ca-4dab-89e3-d3d02f4264a9" containerName="revoke-keys"
Oct 31 01:13:57.375341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.375290    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e88ecd35-15ca-4dab-89e3-d3d02f4264a9" containerName="revoke-keys"
Oct 31 01:13:57.375446 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.375437    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e88ecd35-15ca-4dab-89e3-d3d02f4264a9" containerName="revoke-keys"
Oct 31 01:13:57.379445 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.379416    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.384324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.384290    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"]
Oct 31 01:13:57.392278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.392098    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.393683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.393665    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"]
Oct 31 01:13:57.406649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.402217    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"]
Oct 31 01:13:57.574421 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.574345    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.574566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.574443    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmz98\" (UniqueName: \"kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.574566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.574498    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.574566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.574516    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.574786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.574766    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.574828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.574810    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpf9p\" (UniqueName: \"kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.669840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.669813    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"
Oct 31 01:13:57.670196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.669848    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp" event={"ID":"e88ecd35-15ca-4dab-89e3-d3d02f4264a9","Type":"ContainerDied","Data":"e12a8701305e8703868fd8dab13c4be5e010c4aab6557e8c65a99ded33532373"}
Oct 31 01:13:57.670196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.669895    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e12a8701305e8703868fd8dab13c4be5e010c4aab6557e8c65a99ded33532373"
Oct 31 01:13:57.676128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676102    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.676277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676157    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmz98\" (UniqueName: \"kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.676277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676226    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.676277 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676253    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.676455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676431    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.676619 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676487    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpf9p\" (UniqueName: \"kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.676878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676851    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.677001 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.676975    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.679550 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.679448    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.681931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.681908    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.685528 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.685438    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"]
Oct 31 01:13:57.701296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.692075    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmz98\" (UniqueName: \"kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98\") pod \"maas-api-revoke-sub-dbdb7259-6tlt6\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.701296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.698210    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpf9p\" (UniqueName: \"kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p\") pod \"maas-api-revoke-sub-c61cb88f-j4kmc\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.703450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.701526    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:13:57.703450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.703269    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"]
Oct 31 01:13:57.703450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.703365    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.723725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.723700    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:13:57.779064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.779032    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmg6z\" (UniqueName: \"kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.779325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.779300    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.779404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.779356    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.915324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.891582    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmg6z\" (UniqueName: \"kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.915324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.892759    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.915324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.892824    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.915324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.893580    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.916526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.916489    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.932715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.932688    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmg6z\" (UniqueName: \"kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z\") pod \"maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:57.998948 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:57.998912    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:58.033384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.033355    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:13:58.097365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.097278    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v\" (UniqueName: \"kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v\") pod \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") "
Oct 31 01:13:58.097365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.097346    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca\") pod \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") "
Oct 31 01:13:58.098009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.097987    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca\") pod \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\" (UID: \"f47907ba-7a0c-4fd1-860c-7d1bccde84be\") "
Oct 31 01:13:58.098779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.098753    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca" pod "f47907ba-7a0c-4fd1-860c-7d1bccde84be" (UID: "f47907ba-7a0c-4fd1-860c-7d1bccde84be"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:13:58.099959 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.099935    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca" pod "f47907ba-7a0c-4fd1-860c-7d1bccde84be" (UID: "f47907ba-7a0c-4fd1-860c-7d1bccde84be"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:13:58.100096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.100069    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v" pod "f47907ba-7a0c-4fd1-860c-7d1bccde84be" (UID: "f47907ba-7a0c-4fd1-860c-7d1bccde84be"). InnerVolumeSpecName "kube-api-access-pqj2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:13:58.200558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.200357    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:58.200558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.200391    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqj2v\" (UniqueName: \"kubernetes.io/projected/f47907ba-7a0c-4fd1-860c-7d1bccde84be-kube-api-access-pqj2v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:58.200558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.200407    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f47907ba-7a0c-4fd1-860c-7d1bccde84be-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:13:58.230329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.230298    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"]
Oct 31 01:13:58.230423 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:58.230334    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb2c902_5b24_4658_afdd_a9e853d5857e.slice/crio-3db94c75502d9f69efbb2b579c2f863896186dbfe35bcc9163c703126a1a9a78 WatchSource:0}: Error finding container 3db94c75502d9f69efbb2b579c2f863896186dbfe35bcc9163c703126a1a9a78: Status 404 returned error can't find the container with id 3db94c75502d9f69efbb2b579c2f863896186dbfe35bcc9163c703126a1a9a78
Oct 31 01:13:58.260662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.260637    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"]
Oct 31 01:13:58.266941 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:58.266918    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b70060f_9a58_491d_a2cb_83349d787957.slice/crio-91fa07f1a90aa2e73410bd4b5c3af56f868694308dd4f63c1c44e926b3af89ad WatchSource:0}: Error finding container 91fa07f1a90aa2e73410bd4b5c3af56f868694308dd4f63c1c44e926b3af89ad: Status 404 returned error can't find the container with id 91fa07f1a90aa2e73410bd4b5c3af56f868694308dd4f63c1c44e926b3af89ad
Oct 31 01:13:58.333189 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:13:58.333155    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58689c65_3e8b_4e91_bdb6_5d4d71fec223.slice/crio-7df38cde3fee74461444eab221ca8b9e4e7b7ec6056bcd0eb3a55fc8bc5172a8 WatchSource:0}: Error finding container 7df38cde3fee74461444eab221ca8b9e4e7b7ec6056bcd0eb3a55fc8bc5172a8: Status 404 returned error can't find the container with id 7df38cde3fee74461444eab221ca8b9e4e7b7ec6056bcd0eb3a55fc8bc5172a8
Oct 31 01:13:58.334075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.334054    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"]
Oct 31 01:13:58.679973 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.679869    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr" event={"ID":"58689c65-3e8b-4e91-bdb6-5d4d71fec223","Type":"ContainerStarted","Data":"7df38cde3fee74461444eab221ca8b9e4e7b7ec6056bcd0eb3a55fc8bc5172a8"}
Oct 31 01:13:58.681855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.681823    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6" event={"ID":"abb2c902-5b24-4658-afdd-a9e853d5857e","Type":"ContainerStarted","Data":"44feb80a579528badbc3fc510b8476544c6f2658ab3508b6db9cabe3d669b350"}
Oct 31 01:13:58.681980 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.681876    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6" event={"ID":"abb2c902-5b24-4658-afdd-a9e853d5857e","Type":"ContainerStarted","Data":"3db94c75502d9f69efbb2b579c2f863896186dbfe35bcc9163c703126a1a9a78"}
Oct 31 01:13:58.683336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.683311    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc" event={"ID":"0b70060f-9a58-491d-a2cb-83349d787957","Type":"ContainerStarted","Data":"91fa07f1a90aa2e73410bd4b5c3af56f868694308dd4f63c1c44e926b3af89ad"}
Oct 31 01:13:58.684753 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.684723    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc" event={"ID":"f47907ba-7a0c-4fd1-860c-7d1bccde84be","Type":"ContainerDied","Data":"de9e927c7280737ea3c8baab978288220708abaf7c5dfc58ca9df3fe55666698"}
Oct 31 01:13:58.684753 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.684748    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"
Oct 31 01:13:58.684895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.684776    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de9e927c7280737ea3c8baab978288220708abaf7c5dfc58ca9df3fe55666698"
Oct 31 01:13:58.702345 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:58.702305    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6" podStartSLOduration=1.7022921210000002 podStartE2EDuration="1.702292121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:13:58.696406856 +0000 UTC m=+1035.683518832" watchObservedRunningTime="2026-10-31 01:13:58.702292121 +0000 UTC m=+1035.689404096"
Oct 31 01:13:59.695782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:59.695747    8029 generic.go:356] "Generic (PLEG): container finished" podID="58689c65-3e8b-4e91-bdb6-5d4d71fec223" containerID="da1e693c4a7ddf47f5f61c6a40986b0446a258a4e5691d80101e02ef10e61398" exitCode=0
Oct 31 01:13:59.696266 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:59.695835    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr" event={"ID":"58689c65-3e8b-4e91-bdb6-5d4d71fec223","Type":"ContainerDied","Data":"da1e693c4a7ddf47f5f61c6a40986b0446a258a4e5691d80101e02ef10e61398"}
Oct 31 01:13:59.697944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:59.697920    8029 generic.go:356] "Generic (PLEG): container finished" podID="abb2c902-5b24-4658-afdd-a9e853d5857e" containerID="44feb80a579528badbc3fc510b8476544c6f2658ab3508b6db9cabe3d669b350" exitCode=0
Oct 31 01:13:59.698076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:59.697982    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6" event={"ID":"abb2c902-5b24-4658-afdd-a9e853d5857e","Type":"ContainerDied","Data":"44feb80a579528badbc3fc510b8476544c6f2658ab3508b6db9cabe3d669b350"}
Oct 31 01:13:59.700314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:59.700277    8029 generic.go:356] "Generic (PLEG): container finished" podID="0b70060f-9a58-491d-a2cb-83349d787957" containerID="e080cd0d4ad49af0f4927e0966b3c12421cf74bcfa84acec1d2eb729fbc77603" exitCode=0
Oct 31 01:13:59.700427 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:13:59.700337    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc" event={"ID":"0b70060f-9a58-491d-a2cb-83349d787957","Type":"ContainerDied","Data":"e080cd0d4ad49af0f4927e0966b3c12421cf74bcfa84acec1d2eb729fbc77603"}
Oct 31 01:14:00.945288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:00.945260    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:14:01.010315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.010289    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:14:01.015169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.015145    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:14:01.059070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059040    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca\") pod \"0b70060f-9a58-491d-a2cb-83349d787957\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") "
Oct 31 01:14:01.059231 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059178    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca\") pod \"0b70060f-9a58-491d-a2cb-83349d787957\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") "
Oct 31 01:14:01.059231 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059224    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca\") pod \"abb2c902-5b24-4658-afdd-a9e853d5857e\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") "
Oct 31 01:14:01.059359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059286    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca\") pod \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") "
Oct 31 01:14:01.059359 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059335    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98\" (UniqueName: \"kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98\") pod \"abb2c902-5b24-4658-afdd-a9e853d5857e\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") "
Oct 31 01:14:01.059480 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059382    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p\" (UniqueName: \"kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p\") pod \"0b70060f-9a58-491d-a2cb-83349d787957\" (UID: \"0b70060f-9a58-491d-a2cb-83349d787957\") "
Oct 31 01:14:01.059572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059556    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca\") pod \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") "
Oct 31 01:14:01.059660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059591    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z\" (UniqueName: \"kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z\") pod \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\" (UID: \"58689c65-3e8b-4e91-bdb6-5d4d71fec223\") "
Oct 31 01:14:01.059741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.059726    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca\") pod \"abb2c902-5b24-4658-afdd-a9e853d5857e\" (UID: \"abb2c902-5b24-4658-afdd-a9e853d5857e\") "
Oct 31 01:14:01.060981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.060956    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca" pod "58689c65-3e8b-4e91-bdb6-5d4d71fec223" (UID: "58689c65-3e8b-4e91-bdb6-5d4d71fec223"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:01.062495 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.062467    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca" pod "0b70060f-9a58-491d-a2cb-83349d787957" (UID: "0b70060f-9a58-491d-a2cb-83349d787957"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:01.062715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.062638    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca" pod "abb2c902-5b24-4658-afdd-a9e853d5857e" (UID: "abb2c902-5b24-4658-afdd-a9e853d5857e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:01.069027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.069004    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98" pod "abb2c902-5b24-4658-afdd-a9e853d5857e" (UID: "abb2c902-5b24-4658-afdd-a9e853d5857e"). InnerVolumeSpecName "kube-api-access-fmz98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:01.069145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.069033    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z" pod "58689c65-3e8b-4e91-bdb6-5d4d71fec223" (UID: "58689c65-3e8b-4e91-bdb6-5d4d71fec223"). InnerVolumeSpecName "kube-api-access-zmg6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:01.069145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.069126    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p" pod "0b70060f-9a58-491d-a2cb-83349d787957" (UID: "0b70060f-9a58-491d-a2cb-83349d787957"). InnerVolumeSpecName "kube-api-access-dpf9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:01.069262 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.069230    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca" pod "abb2c902-5b24-4658-afdd-a9e853d5857e" (UID: "abb2c902-5b24-4658-afdd-a9e853d5857e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:01.069566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.069542    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca" pod "58689c65-3e8b-4e91-bdb6-5d4d71fec223" (UID: "58689c65-3e8b-4e91-bdb6-5d4d71fec223"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:01.070932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.070901    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca" pod "0b70060f-9a58-491d-a2cb-83349d787957" (UID: "0b70060f-9a58-491d-a2cb-83349d787957"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:01.163087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163054    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163085    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163095    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b70060f-9a58-491d-a2cb-83349d787957-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163105    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/abb2c902-5b24-4658-afdd-a9e853d5857e-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163116    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163124    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmz98\" (UniqueName: \"kubernetes.io/projected/abb2c902-5b24-4658-afdd-a9e853d5857e-kube-api-access-fmz98\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163132    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpf9p\" (UniqueName: \"kubernetes.io/projected/0b70060f-9a58-491d-a2cb-83349d787957-kube-api-access-dpf9p\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163141    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/58689c65-3e8b-4e91-bdb6-5d4d71fec223-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.163334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.163149    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmg6z\" (UniqueName: \"kubernetes.io/projected/58689c65-3e8b-4e91-bdb6-5d4d71fec223-kube-api-access-zmg6z\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:01.773379 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.773325    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr" event={"ID":"58689c65-3e8b-4e91-bdb6-5d4d71fec223","Type":"ContainerDied","Data":"7df38cde3fee74461444eab221ca8b9e4e7b7ec6056bcd0eb3a55fc8bc5172a8"}
Oct 31 01:14:01.773546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.773443    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df38cde3fee74461444eab221ca8b9e4e7b7ec6056bcd0eb3a55fc8bc5172a8"
Oct 31 01:14:01.773546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.773529    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"
Oct 31 01:14:01.777126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.777101    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"
Oct 31 01:14:01.777378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.777146    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6" event={"ID":"abb2c902-5b24-4658-afdd-a9e853d5857e","Type":"ContainerDied","Data":"3db94c75502d9f69efbb2b579c2f863896186dbfe35bcc9163c703126a1a9a78"}
Oct 31 01:14:01.777459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.777404    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3db94c75502d9f69efbb2b579c2f863896186dbfe35bcc9163c703126a1a9a78"
Oct 31 01:14:01.779974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.779939    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc" event={"ID":"0b70060f-9a58-491d-a2cb-83349d787957","Type":"ContainerDied","Data":"91fa07f1a90aa2e73410bd4b5c3af56f868694308dd4f63c1c44e926b3af89ad"}
Oct 31 01:14:01.780109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.779989    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91fa07f1a90aa2e73410bd4b5c3af56f868694308dd4f63c1c44e926b3af89ad"
Oct 31 01:14:01.780109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:01.780025    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"
Oct 31 01:14:04.012400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.012368    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"]
Oct 31 01:14:04.013111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013096    8029 cpu_manager.go:414] "containerMap: removing container" podUID="58689c65-3e8b-4e91-bdb6-5d4d71fec223" containerName="revoke-keys"
Oct 31 01:14:04.013151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013113    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58689c65-3e8b-4e91-bdb6-5d4d71fec223" containerName="revoke-keys"
Oct 31 01:14:04.013151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013122    8029 cpu_manager.go:414] "containerMap: removing container" podUID="abb2c902-5b24-4658-afdd-a9e853d5857e" containerName="revoke-keys"
Oct 31 01:14:04.013151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013128    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abb2c902-5b24-4658-afdd-a9e853d5857e" containerName="revoke-keys"
Oct 31 01:14:04.013151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013138    8029 cpu_manager.go:414] "containerMap: removing container" podUID="0b70060f-9a58-491d-a2cb-83349d787957" containerName="revoke-keys"
Oct 31 01:14:04.013151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013142    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b70060f-9a58-491d-a2cb-83349d787957" containerName="revoke-keys"
Oct 31 01:14:04.013313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013162    8029 cpu_manager.go:414] "containerMap: removing container" podUID="f47907ba-7a0c-4fd1-860c-7d1bccde84be" containerName="revoke-keys"
Oct 31 01:14:04.013313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013167    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f47907ba-7a0c-4fd1-860c-7d1bccde84be" containerName="revoke-keys"
Oct 31 01:14:04.013313 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013306    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="58689c65-3e8b-4e91-bdb6-5d4d71fec223" containerName="revoke-keys"
Oct 31 01:14:04.013394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013315    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="abb2c902-5b24-4658-afdd-a9e853d5857e" containerName="revoke-keys"
Oct 31 01:14:04.013394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013323    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b70060f-9a58-491d-a2cb-83349d787957" containerName="revoke-keys"
Oct 31 01:14:04.013394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.013332    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f47907ba-7a0c-4fd1-860c-7d1bccde84be" containerName="revoke-keys"
Oct 31 01:14:04.018474 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.018266    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.022505 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.022483    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:14:04.022619 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.022574    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:14:04.022619 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.022614    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:14:04.022705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.022675    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:14:04.026980 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.026955    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"]
Oct 31 01:14:04.129245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.129213    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx2km\" (UniqueName: \"kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.129414 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.129336    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.129587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.129568    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.235740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.235707    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.236032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.235994    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx2km\" (UniqueName: \"kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.236167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.236082    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.236649 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.236577    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.238732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.238715    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.244476 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.244453    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx2km\" (UniqueName: \"kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km\") pod \"maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.337371 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.337277    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:04.633910 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.633870    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"]
Oct 31 01:14:04.805173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:04.805138    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465" event={"ID":"54cb07c4-233e-46d0-80df-57815272239b","Type":"ContainerStarted","Data":"329f0cb6aaf0d8adaebdc6c84988d169ec12ff75a31987c0911835fed3afc25c"}
Oct 31 01:14:05.816219 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:05.816179    8029 generic.go:356] "Generic (PLEG): container finished" podID="54cb07c4-233e-46d0-80df-57815272239b" containerID="b5395b605e857cc7f99fd4d41ffedc49f9268269a2402632d3db287493179062" exitCode=0
Oct 31 01:14:05.816635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:05.816265    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465" event={"ID":"54cb07c4-233e-46d0-80df-57815272239b","Type":"ContainerDied","Data":"b5395b605e857cc7f99fd4d41ffedc49f9268269a2402632d3db287493179062"}
Oct 31 01:14:06.171292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.171260    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"]
Oct 31 01:14:06.176367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.176346    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.184920 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.184892    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"]
Oct 31 01:14:06.275764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.275731    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.275947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.275837    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.275996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.275983    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4tcd\" (UniqueName: \"kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.377448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.377412    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.377644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.377466    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.377644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.377547    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4tcd\" (UniqueName: \"kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.378272 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.378247    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.379891 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.379870    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.385304 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.385283    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4tcd\" (UniqueName: \"kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd\") pod \"maas-api-revoke-sub-627c06ba-xhkkc\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.494745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.494656    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:06.730901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.730869    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"]
Oct 31 01:14:06.735970 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.735941    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.745414 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.745343    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"]
Oct 31 01:14:06.786562 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:06.786530    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74cbf15_4be0_4b33_9e40_d36950c640ec.slice/crio-7a8cf8b0f49ea5609f8d3510cec820ccfc312cb3f4c35f78447656dcf31d4f3d WatchSource:0}: Error finding container 7a8cf8b0f49ea5609f8d3510cec820ccfc312cb3f4c35f78447656dcf31d4f3d: Status 404 returned error can't find the container with id 7a8cf8b0f49ea5609f8d3510cec820ccfc312cb3f4c35f78447656dcf31d4f3d
Oct 31 01:14:06.788259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.788086    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"]
Oct 31 01:14:06.826675 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.826637    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc" event={"ID":"a74cbf15-4be0-4b33-9e40-d36950c640ec","Type":"ContainerStarted","Data":"7a8cf8b0f49ea5609f8d3510cec820ccfc312cb3f4c35f78447656dcf31d4f3d"}
Oct 31 01:14:06.886572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.886543    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x72gc\" (UniqueName: \"kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.886808 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.886581    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.886808 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.886966    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.989251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.989205    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.989448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.989430    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x72gc\" (UniqueName: \"kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.989586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.989467    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.990169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.990128    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:06.990337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.990155    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:06.993090 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.992893    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:07.001385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:06.998434    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x72gc\" (UniqueName: \"kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc\") pod \"maas-api-revoke-sub-7bad7863-kjpv8\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:07.059773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.059736    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:07.090762 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.090727    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca\") pod \"54cb07c4-233e-46d0-80df-57815272239b\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") "
Oct 31 01:14:07.090899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.090774    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca\") pod \"54cb07c4-233e-46d0-80df-57815272239b\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") "
Oct 31 01:14:07.090899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.090873    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km\" (UniqueName: \"kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km\") pod \"54cb07c4-233e-46d0-80df-57815272239b\" (UID: \"54cb07c4-233e-46d0-80df-57815272239b\") "
Oct 31 01:14:07.091215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.091189    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca" pod "54cb07c4-233e-46d0-80df-57815272239b" (UID: "54cb07c4-233e-46d0-80df-57815272239b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:07.091615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.091582    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54cb07c4-233e-46d0-80df-57815272239b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:07.092997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.092968    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca" pod "54cb07c4-233e-46d0-80df-57815272239b" (UID: "54cb07c4-233e-46d0-80df-57815272239b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:07.093147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.093127    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km" pod "54cb07c4-233e-46d0-80df-57815272239b" (UID: "54cb07c4-233e-46d0-80df-57815272239b"). InnerVolumeSpecName "kube-api-access-gx2km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:07.194115 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.194085    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54cb07c4-233e-46d0-80df-57815272239b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:07.194227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.194122    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx2km\" (UniqueName: \"kubernetes.io/projected/54cb07c4-233e-46d0-80df-57815272239b-kube-api-access-gx2km\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:07.358517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.358497    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"]
Oct 31 01:14:07.361299 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:07.361267    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8c649f_ed6e_4d5c_9f83_959d51b79706.slice/crio-dbdb96b4e332b9446b99ffe359165155100199a2e0d50e988d2557952075855c WatchSource:0}: Error finding container dbdb96b4e332b9446b99ffe359165155100199a2e0d50e988d2557952075855c: Status 404 returned error can't find the container with id dbdb96b4e332b9446b99ffe359165155100199a2e0d50e988d2557952075855c
Oct 31 01:14:07.838624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.837979    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8" event={"ID":"df8c649f-ed6e-4d5c-9f83-959d51b79706","Type":"ContainerStarted","Data":"5091e41d389d176235945ca3c80a2b9341a283429b05fed0610e870393350a48"}
Oct 31 01:14:07.838624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.838085    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8" event={"ID":"df8c649f-ed6e-4d5c-9f83-959d51b79706","Type":"ContainerStarted","Data":"dbdb96b4e332b9446b99ffe359165155100199a2e0d50e988d2557952075855c"}
Oct 31 01:14:07.841625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.839703    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"
Oct 31 01:14:07.841625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.839743    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465" event={"ID":"54cb07c4-233e-46d0-80df-57815272239b","Type":"ContainerDied","Data":"329f0cb6aaf0d8adaebdc6c84988d169ec12ff75a31987c0911835fed3afc25c"}
Oct 31 01:14:07.841625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.839798    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="329f0cb6aaf0d8adaebdc6c84988d169ec12ff75a31987c0911835fed3afc25c"
Oct 31 01:14:07.841862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.841748    8029 generic.go:356] "Generic (PLEG): container finished" podID="a74cbf15-4be0-4b33-9e40-d36950c640ec" containerID="981ac009644edec54bf3ccd8d1a34528cf5af102b1244749ba3a0ee104f0202c" exitCode=0
Oct 31 01:14:07.841862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.841784    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc" event={"ID":"a74cbf15-4be0-4b33-9e40-d36950c640ec","Type":"ContainerDied","Data":"981ac009644edec54bf3ccd8d1a34528cf5af102b1244749ba3a0ee104f0202c"}
Oct 31 01:14:07.859914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.859872    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8" podStartSLOduration=1.859860334 podStartE2EDuration="1.859860334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:07.855015784 +0000 UTC m=+1044.842127761" watchObservedRunningTime="2026-10-31 01:14:07.859860334 +0000 UTC m=+1044.846972339"
Oct 31 01:14:07.883427 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:07.883391    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"]
Oct 31 01:14:08.111438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.111408    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"]
Oct 31 01:14:08.112256 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.112237    8029 cpu_manager.go:414] "containerMap: removing container" podUID="54cb07c4-233e-46d0-80df-57815272239b" containerName="revoke-keys"
Oct 31 01:14:08.112346 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.112257    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54cb07c4-233e-46d0-80df-57815272239b" containerName="revoke-keys"
Oct 31 01:14:08.112399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.112379    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="54cb07c4-233e-46d0-80df-57815272239b" containerName="revoke-keys"
Oct 31 01:14:08.116430 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.116401    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.128795 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.128709    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"]
Oct 31 01:14:08.163978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.161510    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"]
Oct 31 01:14:08.163978 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.161692    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.427502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.427465    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"]
Oct 31 01:14:08.433232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.429792    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blpdq\" (UniqueName: \"kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.433232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.429878    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.433232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.429913    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.433232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.429953    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2v76\" (UniqueName: \"kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.433232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.430200    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.433232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.430352    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.531777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.531744    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.531947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.531848    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blpdq\" (UniqueName: \"kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.531947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.531923    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.532033 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.531962    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.532033 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.532010    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2v76\" (UniqueName: \"kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.532271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.532250    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.532704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.532679    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.533149 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.533125    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.535010 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.534990    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.535186 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.535163    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.540527 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.540505    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blpdq\" (UniqueName: \"kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq\") pod \"maas-api-revoke-sub-5f834682-ndbwc\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.541470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.541445    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2v76\" (UniqueName: \"kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76\") pod \"maas-api-revoke-sub-9ee0c41c-68rgc\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.639664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.639493    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:08.766585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.766549    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:08.794520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.794489    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"]
Oct 31 01:14:08.861442 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.861407    8029 generic.go:356] "Generic (PLEG): container finished" podID="df8c649f-ed6e-4d5c-9f83-959d51b79706" containerID="5091e41d389d176235945ca3c80a2b9341a283429b05fed0610e870393350a48" exitCode=0
Oct 31 01:14:08.862470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:08.862443    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8" event={"ID":"df8c649f-ed6e-4d5c-9f83-959d51b79706","Type":"ContainerDied","Data":"5091e41d389d176235945ca3c80a2b9341a283429b05fed0610e870393350a48"}
Oct 31 01:14:09.017184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.017115    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"]
Oct 31 01:14:09.115926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.115738    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"]
Oct 31 01:14:09.347706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.347682    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:09.406329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.406292    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca\") pod \"a74cbf15-4be0-4b33-9e40-d36950c640ec\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") "
Oct 31 01:14:09.406517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.406392    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca\") pod \"a74cbf15-4be0-4b33-9e40-d36950c640ec\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") "
Oct 31 01:14:09.406569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.406549    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd\" (UniqueName: \"kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd\") pod \"a74cbf15-4be0-4b33-9e40-d36950c640ec\" (UID: \"a74cbf15-4be0-4b33-9e40-d36950c640ec\") "
Oct 31 01:14:09.416102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.416057    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca" pod "a74cbf15-4be0-4b33-9e40-d36950c640ec" (UID: "a74cbf15-4be0-4b33-9e40-d36950c640ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:09.417745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.417711    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd" pod "a74cbf15-4be0-4b33-9e40-d36950c640ec" (UID: "a74cbf15-4be0-4b33-9e40-d36950c640ec"). InnerVolumeSpecName "kube-api-access-f4tcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:09.418103 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.418082    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca" pod "a74cbf15-4be0-4b33-9e40-d36950c640ec" (UID: "a74cbf15-4be0-4b33-9e40-d36950c640ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:09.470820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.470786    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"]
Oct 31 01:14:09.476327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.476302    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-1b8d05-k6wvp"]
Oct 31 01:14:09.512735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.512659    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:09.512735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.512690    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a74cbf15-4be0-4b33-9e40-d36950c640ec-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:09.512735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.512707    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4tcd\" (UniqueName: \"kubernetes.io/projected/a74cbf15-4be0-4b33-9e40-d36950c640ec-kube-api-access-f4tcd\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:09.871971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.871900    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc" event={"ID":"25f965ef-9029-46ca-af46-7503825eaafb","Type":"ContainerStarted","Data":"a59b0cb8b5b65ba96a3c01dc2473c68c7091c25bf4593a674daa10aadea2287a"}
Oct 31 01:14:09.872411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.871994    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc" event={"ID":"25f965ef-9029-46ca-af46-7503825eaafb","Type":"ContainerStarted","Data":"fda751bc6976bf107808934c340f7cf1650e9a566c6cdbb7dc44da8f1fd2b785"}
Oct 31 01:14:09.873642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.873594    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc" event={"ID":"8e1dc8f5-0b89-4577-968b-c846879dc0ff","Type":"ContainerStarted","Data":"1c5ab74ab75f43e37e877b911b886b5c7b4bd0c8e42c7d60f394006753526a98"}
Oct 31 01:14:09.873763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.873669    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc" event={"ID":"8e1dc8f5-0b89-4577-968b-c846879dc0ff","Type":"ContainerStarted","Data":"099f63d5fad2a4269486c51abd047649060ce7cf2c150c388f0025770008a4e9"}
Oct 31 01:14:09.875078 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.875060    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"
Oct 31 01:14:09.875167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.875090    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc" event={"ID":"a74cbf15-4be0-4b33-9e40-d36950c640ec","Type":"ContainerDied","Data":"7a8cf8b0f49ea5609f8d3510cec820ccfc312cb3f4c35f78447656dcf31d4f3d"}
Oct 31 01:14:09.875167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.875128    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a8cf8b0f49ea5609f8d3510cec820ccfc312cb3f4c35f78447656dcf31d4f3d"
Oct 31 01:14:09.890871 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.890828    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc" podStartSLOduration=1.8908143929999999 podStartE2EDuration="1.890814393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:09.889706187 +0000 UTC m=+1046.876818165" watchObservedRunningTime="2026-10-31 01:14:09.890814393 +0000 UTC m=+1046.877926368"
Oct 31 01:14:09.916311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:09.916264    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc" podStartSLOduration=1.9162409390000001 podStartE2EDuration="1.916240939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:09.912113671 +0000 UTC m=+1046.899225651" watchObservedRunningTime="2026-10-31 01:14:09.916240939 +0000 UTC m=+1046.903352938"
Oct 31 01:14:10.065726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.065683    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:10.124444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.124358    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca\") pod \"df8c649f-ed6e-4d5c-9f83-959d51b79706\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") "
Oct 31 01:14:10.124618 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.124497    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca\") pod \"df8c649f-ed6e-4d5c-9f83-959d51b79706\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") "
Oct 31 01:14:10.124681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.124657    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc\" (UniqueName: \"kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc\") pod \"df8c649f-ed6e-4d5c-9f83-959d51b79706\" (UID: \"df8c649f-ed6e-4d5c-9f83-959d51b79706\") "
Oct 31 01:14:10.124879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.124848    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca" pod "df8c649f-ed6e-4d5c-9f83-959d51b79706" (UID: "df8c649f-ed6e-4d5c-9f83-959d51b79706"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:10.125371 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.125350    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:10.126881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.126857    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc" pod "df8c649f-ed6e-4d5c-9f83-959d51b79706" (UID: "df8c649f-ed6e-4d5c-9f83-959d51b79706"). InnerVolumeSpecName "kube-api-access-x72gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:10.127257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.127239    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca" pod "df8c649f-ed6e-4d5c-9f83-959d51b79706" (UID: "df8c649f-ed6e-4d5c-9f83-959d51b79706"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:10.226813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.226769    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df8c649f-ed6e-4d5c-9f83-959d51b79706-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:10.226813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.226796    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x72gc\" (UniqueName: \"kubernetes.io/projected/df8c649f-ed6e-4d5c-9f83-959d51b79706-kube-api-access-x72gc\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:10.887225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.887197    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"
Oct 31 01:14:10.887225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.887227    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8" event={"ID":"df8c649f-ed6e-4d5c-9f83-959d51b79706","Type":"ContainerDied","Data":"dbdb96b4e332b9446b99ffe359165155100199a2e0d50e988d2557952075855c"}
Oct 31 01:14:10.887735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.887286    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbdb96b4e332b9446b99ffe359165155100199a2e0d50e988d2557952075855c"
Oct 31 01:14:10.889292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.889266    8029 generic.go:356] "Generic (PLEG): container finished" podID="25f965ef-9029-46ca-af46-7503825eaafb" containerID="a59b0cb8b5b65ba96a3c01dc2473c68c7091c25bf4593a674daa10aadea2287a" exitCode=0
Oct 31 01:14:10.889440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.889332    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc" event={"ID":"25f965ef-9029-46ca-af46-7503825eaafb","Type":"ContainerDied","Data":"a59b0cb8b5b65ba96a3c01dc2473c68c7091c25bf4593a674daa10aadea2287a"}
Oct 31 01:14:10.891385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.891359    8029 generic.go:356] "Generic (PLEG): container finished" podID="8e1dc8f5-0b89-4577-968b-c846879dc0ff" containerID="1c5ab74ab75f43e37e877b911b886b5c7b4bd0c8e42c7d60f394006753526a98" exitCode=0
Oct 31 01:14:10.891583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:10.891448    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc" event={"ID":"8e1dc8f5-0b89-4577-968b-c846879dc0ff","Type":"ContainerDied","Data":"1c5ab74ab75f43e37e877b911b886b5c7b4bd0c8e42c7d60f394006753526a98"}
Oct 31 01:14:11.203677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.203327    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4898e302-d834-49c5-ada3-f507d6dd6889" path="/var/lib/kubelet/pods/4898e302-d834-49c5-ada3-f507d6dd6889/volumes"
Oct 31 01:14:11.609206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.609167    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"]
Oct 31 01:14:11.612043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.611330    8029 cpu_manager.go:414] "containerMap: removing container" podUID="df8c649f-ed6e-4d5c-9f83-959d51b79706" containerName="revoke-keys"
Oct 31 01:14:11.612043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.611357    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df8c649f-ed6e-4d5c-9f83-959d51b79706" containerName="revoke-keys"
Oct 31 01:14:11.612043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.611374    8029 cpu_manager.go:414] "containerMap: removing container" podUID="a74cbf15-4be0-4b33-9e40-d36950c640ec" containerName="revoke-keys"
Oct 31 01:14:11.612043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.611381    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a74cbf15-4be0-4b33-9e40-d36950c640ec" containerName="revoke-keys"
Oct 31 01:14:11.612043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.611639    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="df8c649f-ed6e-4d5c-9f83-959d51b79706" containerName="revoke-keys"
Oct 31 01:14:11.612043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.611656    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a74cbf15-4be0-4b33-9e40-d36950c640ec" containerName="revoke-keys"
Oct 31 01:14:11.634654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.616412    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.634654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.620245    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"]
Oct 31 01:14:11.784627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.783656    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4rlv\" (UniqueName: \"kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.784627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.783936    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.784627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.784044    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.896673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.891997    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.896673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.892106    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4rlv\" (UniqueName: \"kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.896673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.892411    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.896673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.893306    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:11.969633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:11.969578    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4rlv\" (UniqueName: \"kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:12.022636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:12.004678    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca\") pod \"maas-api-revoke-sub-12a087ad-zlmb6\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:12.292708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:12.292593    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:12.925117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:12.925012    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" containerName="payload-processing" containerID="cri-o://eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb" gracePeriod=25
Oct 31 01:14:12.926186 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:12.926152    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/health\": context deadline exceeded"
Oct 31 01:14:13.736365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:13.736324    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:14:13.924411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:13.920812    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:13.951628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:13.926644    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:13.978625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:13.953433    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:14.162263 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:14.162214    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:14.723684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:14.714288    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:14.838500 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:14.838450    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:15.084276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:14.871297    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:14:15.085567 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:14.989125    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76\" (UniqueName: \"kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76\") pod \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") "
Oct 31 01:14:15.109538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.109490    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca\") pod \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") "
Oct 31 01:14:15.109899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.109879    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca\") pod \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\" (UID: \"8e1dc8f5-0b89-4577-968b-c846879dc0ff\") "
Oct 31 01:14:15.119523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.068716    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:15.186545 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.186498    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:14:15.203776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.203729    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:14:15.221767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.221716    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:15.222040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.222014    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:14:15.279647 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.248467    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:14:15.317725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.249105    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:15.317725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.289046    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:15.317725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.265806    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca" pod "8e1dc8f5-0b89-4577-968b-c846879dc0ff" (UID: "8e1dc8f5-0b89-4577-968b-c846879dc0ff"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:15.317725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.308896    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:15.317725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.310341    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": context deadline exceeded"
Oct 31 01:14:15.317725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.310426    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:14:15.700183 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.551198    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:15.877644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.759909    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:15.942642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:15.919963    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.011679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.005239    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca\") pod \"25f965ef-9029-46ca-af46-7503825eaafb\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") "
Oct 31 01:14:16.011679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.005308    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca\") pod \"25f965ef-9029-46ca-af46-7503825eaafb\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") "
Oct 31 01:14:16.011679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.005646    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq\" (UniqueName: \"kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq\") pod \"25f965ef-9029-46ca-af46-7503825eaafb\" (UID: \"25f965ef-9029-46ca-af46-7503825eaafb\") "
Oct 31 01:14:16.131630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.116731    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca" pod "25f965ef-9029-46ca-af46-7503825eaafb" (UID: "25f965ef-9029-46ca-af46-7503825eaafb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:16.183627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.176721    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:16.207643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.202363    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:14:16.207643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.202771    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:14:16.434817 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.434754    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.507627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.466809    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq" podUID="4ad09d17-5b0f-4b20-9e81-7c0faf104aae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.507627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.491443    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq" pod "25f965ef-9029-46ca-af46-7503825eaafb" (UID: "25f965ef-9029-46ca-af46-7503825eaafb"). InnerVolumeSpecName "kube-api-access-blpdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:16.507627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.491533    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca" pod "25f965ef-9029-46ca-af46-7503825eaafb" (UID: "25f965ef-9029-46ca-af46-7503825eaafb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:16.629703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.554202    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blpdq\" (UniqueName: \"kubernetes.io/projected/25f965ef-9029-46ca-af46-7503825eaafb-kube-api-access-blpdq\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:16.664654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.659968    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/25f965ef-9029-46ca-af46-7503825eaafb-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:16.664654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.559772    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76" pod "8e1dc8f5-0b89-4577-968b-c846879dc0ff" (UID: "8e1dc8f5-0b89-4577-968b-c846879dc0ff"). InnerVolumeSpecName "kube-api-access-h2v76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:16.664654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.560175    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:16.664654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.660300    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.664654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.595964    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:14:16.668064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.667947    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:16.668064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.668024    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.693627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.692367    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:14:16.693627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.643366    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:14:16.701625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.611461    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:14:16.814864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.643492    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:16.815165 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.815121    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.877812 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.659701    8029 patch_prober.go:29] interesting pod/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:16.878074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.878043    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.879939 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.855106    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2v76\" (UniqueName: \"kubernetes.io/projected/8e1dc8f5-0b89-4577-968b-c846879dc0ff-kube-api-access-h2v76\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:16.883653 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.659734    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:16.883828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.883804    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:16.883942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.881014    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:16.884044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:16.884028    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:17.000407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.000368    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca" pod "8e1dc8f5-0b89-4577-968b-c846879dc0ff" (UID: "8e1dc8f5-0b89-4577-968b-c846879dc0ff"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:17.067453 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.067423    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:17.121273    8029 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.917s"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121333    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"]
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121391    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc" event={"ID":"25f965ef-9029-46ca-af46-7503825eaafb","Type":"ContainerDied","Data":"fda751bc6976bf107808934c340f7cf1650e9a566c6cdbb7dc44da8f1fd2b785"}
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121428    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fda751bc6976bf107808934c340f7cf1650e9a566c6cdbb7dc44da8f1fd2b785"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121486    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121525    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121563    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121618    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121655    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:14:17.130755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.121800    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:14:17.222693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.210271    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 31 01:14:17.297220 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.297158    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:17.328670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.324468    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e1dc8f5-0b89-4577-968b-c846879dc0ff-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:17.481716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.402720    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:17.616631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.566985    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 31 01:14:17.688699 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.683042    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:14:17.943799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.737954    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:14:17.949627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.738225    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:14:17.973624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.738486    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Oct 31 01:14:17.974634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:17.818368    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"
Oct 31 01:14:18.506756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.506709    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:18.635180 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.635122    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q2rrd" podUID="eeda0349-fac8-42fc-a73d-f81c06888419" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.24:6080/livez\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:18.670827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.670763    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:18.671406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.671358    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:18.758166 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:18.758129    8029 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.489s"
Oct 31 01:14:18.758444 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.758416    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc" event={"ID":"8e1dc8f5-0b89-4577-968b-c846879dc0ff","Type":"ContainerDied","Data":"099f63d5fad2a4269486c51abd047649060ce7cf2c150c388f0025770008a4e9"}
Oct 31 01:14:18.758614 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.758578    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="099f63d5fad2a4269486c51abd047649060ce7cf2c150c388f0025770008a4e9"
Oct 31 01:14:18.871635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.866255    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"]
Oct 31 01:14:18.953192 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.945668    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:18.953192 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.946014    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 31 01:14:18.954694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.954663    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:18.957290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.957266    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6" event={"ID":"0b67dbb9-2291-4645-9b71-926b6b936296","Type":"ContainerStarted","Data":"a27fede57469d22d738df63507de04c361ab297e3ce12ae575b4a6ee80ebd6ee"}
Oct 31 01:14:18.957752 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.957730    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6" event={"ID":"0b67dbb9-2291-4645-9b71-926b6b936296","Type":"ContainerStarted","Data":"44fd0cd7e2094c187c034cbe9b561940f0713a17886aaac5dba4a097abffd4ee"}
Oct 31 01:14:18.977116 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:18.977065    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:19.479629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.464194    8029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/health\": context deadline exceeded"
Oct 31 01:14:19.562434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.546706    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:19.636173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.635785    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:19.650380 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.650347    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"]
Oct 31 01:14:19.814652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.767247    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"]
Oct 31 01:14:19.814652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.715913    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": context deadline exceeded"
Oct 31 01:14:19.814652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.715872    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:19.814969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.785336    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy" containerID="cri-o://bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c" gracePeriod=30
Oct 31 01:14:19.958763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.905009    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:19.958763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.905074    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:19.958763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.949910    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:19.958763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.950394    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.85:8000/ready\": context deadline exceeded"
Oct 31 01:14:19.990675 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:19.980088    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:20.042642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.039022    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 31 01:14:20.085930 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.085899    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"]
Oct 31 01:14:20.092932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.092867    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy" containerID="cri-o://5184b2c36b80dc5962ef2a49129e7bc8de84845e483e141b6335d2bc51931ac1" gracePeriod=30
Oct 31 01:14:20.184461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.176983    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy" containerID="cri-o://a6b5accd13235ac4c2ec1f02c72ae82e678fbfa388120ad11c88cc01cc64fb6d" gracePeriod=30
Oct 31 01:14:20.380733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.375574    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"]
Oct 31 01:14:20.380733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.375710    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:14:20.380733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.376380    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy" containerID="cri-o://d75dde75e6f4dfd8bfcabcb0e88cd9e1714d1a2a0d795c07e2dde57333202c56" gracePeriod=30
Oct 31 01:14:20.430575 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.392815    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:14:20.430575 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.402767    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:14:20.440777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.437937    8029 generic.go:356] "Generic (PLEG): container finished" podID="0b67dbb9-2291-4645-9b71-926b6b936296" containerID="a27fede57469d22d738df63507de04c361ab297e3ce12ae575b4a6ee80ebd6ee" exitCode=0
Oct 31 01:14:20.440777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.438256    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6" event={"ID":"0b67dbb9-2291-4645-9b71-926b6b936296","Type":"ContainerDied","Data":"a27fede57469d22d738df63507de04c361ab297e3ce12ae575b4a6ee80ebd6ee"}
Oct 31 01:14:20.548625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.531992    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:14:20.548625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.532405    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:14:20.553496 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.552980    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:14:20.576497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.556064    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:14:20.589509 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:20.589392    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:14:20.695270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.695181    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6" podStartSLOduration=9.69516701 podStartE2EDuration="9.69516701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:19.00281257 +0000 UTC m=+1055.989924544" watchObservedRunningTime="2026-10-31 01:14:20.69516701 +0000 UTC m=+1057.682278977"
Oct 31 01:14:20.751759 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.751709    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"]
Oct 31 01:14:20.753171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.753147    8029 cpu_manager.go:414] "containerMap: removing container" podUID="25f965ef-9029-46ca-af46-7503825eaafb" containerName="revoke-keys"
Oct 31 01:14:20.753339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.753324    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25f965ef-9029-46ca-af46-7503825eaafb" containerName="revoke-keys"
Oct 31 01:14:20.753435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.753425    8029 cpu_manager.go:414] "containerMap: removing container" podUID="8e1dc8f5-0b89-4577-968b-c846879dc0ff" containerName="revoke-keys"
Oct 31 01:14:20.753513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.753502    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e1dc8f5-0b89-4577-968b-c846879dc0ff" containerName="revoke-keys"
Oct 31 01:14:20.753904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.753884    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="25f965ef-9029-46ca-af46-7503825eaafb" containerName="revoke-keys"
Oct 31 01:14:20.754035 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.754024    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e1dc8f5-0b89-4577-968b-c846879dc0ff" containerName="revoke-keys"
Oct 31 01:14:20.767397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.765472    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"]
Oct 31 01:14:20.767397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.765589    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:20.797198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.797172    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"]
Oct 31 01:14:20.811492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.808383    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:20.811492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.809713    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"]
Oct 31 01:14:20.925672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.918483    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:20.925672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.918581    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:20.925672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.918749    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cgd\" (UniqueName: \"kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:20.925672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.918836    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:20.925672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.918868    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:20.925672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:20.918904    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8nv\" (UniqueName: \"kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.020381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.020283    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.020549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.020465    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cgd\" (UniqueName: \"kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.020549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.020538    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.020747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.020560    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.020747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.020591    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8nv\" (UniqueName: \"kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.020868 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.020796    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.021179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.021144    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.021506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.021482    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.023317 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.023293    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.023683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.023665    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.045661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.040300    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cgd\" (UniqueName: \"kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd\") pod \"maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.045661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.043767    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8nv\" (UniqueName: \"kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv\") pod \"maas-api-revoke-sub-b1b23d49-9bflt\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.091700 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.091669    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:21.128677 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.128643    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:21.402868 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:21.402835    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792d3fb7_b6f8_47fc_9017_e2a300023f16.slice/crio-474c3b65e16ac67cd3306aaa78753199573c0ea99e6a1e2f06d7c36e3fc0ba5d WatchSource:0}: Error finding container 474c3b65e16ac67cd3306aaa78753199573c0ea99e6a1e2f06d7c36e3fc0ba5d: Status 404 returned error can't find the container with id 474c3b65e16ac67cd3306aaa78753199573c0ea99e6a1e2f06d7c36e3fc0ba5d
Oct 31 01:14:21.405068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.405037    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"]
Oct 31 01:14:21.443261 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:21.443228    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417b23a9_09ea_4427_8f72_85aad51e5fc6.slice/crio-b3c88efc169b03da47b9933d288c240fa9f6da5ce41ac618fda68ffb6d578ac5 WatchSource:0}: Error finding container b3c88efc169b03da47b9933d288c240fa9f6da5ce41ac618fda68ffb6d578ac5: Status 404 returned error can't find the container with id b3c88efc169b03da47b9933d288c240fa9f6da5ce41ac618fda68ffb6d578ac5
Oct 31 01:14:21.447234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.447208    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"]
Oct 31 01:14:21.452290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.452263    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt" event={"ID":"792d3fb7-b6f8-47fc-9017-e2a300023f16","Type":"ContainerStarted","Data":"474c3b65e16ac67cd3306aaa78753199573c0ea99e6a1e2f06d7c36e3fc0ba5d"}
Oct 31 01:14:21.705850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.705824    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:21.843941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.839580    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv\" (UniqueName: \"kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv\") pod \"0b67dbb9-2291-4645-9b71-926b6b936296\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") "
Oct 31 01:14:21.843941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.839709    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca\") pod \"0b67dbb9-2291-4645-9b71-926b6b936296\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") "
Oct 31 01:14:21.843941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.839809    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca\") pod \"0b67dbb9-2291-4645-9b71-926b6b936296\" (UID: \"0b67dbb9-2291-4645-9b71-926b6b936296\") "
Oct 31 01:14:21.852971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.852924    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca" pod "0b67dbb9-2291-4645-9b71-926b6b936296" (UID: "0b67dbb9-2291-4645-9b71-926b6b936296"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:21.855902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.855843    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv" pod "0b67dbb9-2291-4645-9b71-926b6b936296" (UID: "0b67dbb9-2291-4645-9b71-926b6b936296"). InnerVolumeSpecName "kube-api-access-p4rlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:21.861744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.861681    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca" pod "0b67dbb9-2291-4645-9b71-926b6b936296" (UID: "0b67dbb9-2291-4645-9b71-926b6b936296"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:21.943384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.943290    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4rlv\" (UniqueName: \"kubernetes.io/projected/0b67dbb9-2291-4645-9b71-926b6b936296-kube-api-access-p4rlv\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:21.943384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.943327    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:21.943384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:21.943340    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b67dbb9-2291-4645-9b71-926b6b936296-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:22.611091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.610896    8029 generic.go:356] "Generic (PLEG): container finished" podID="792d3fb7-b6f8-47fc-9017-e2a300023f16" containerID="f9d577698f4ed5f8a4514d2d095952b6463b22eaf5d5f130e00aba74a4506a86" exitCode=0
Oct 31 01:14:22.611259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.611121    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt" event={"ID":"792d3fb7-b6f8-47fc-9017-e2a300023f16","Type":"ContainerDied","Data":"f9d577698f4ed5f8a4514d2d095952b6463b22eaf5d5f130e00aba74a4506a86"}
Oct 31 01:14:22.613891 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.613860    8029 generic.go:356] "Generic (PLEG): container finished" podID="417b23a9-09ea-4427-8f72-85aad51e5fc6" containerID="3df794e5fd30905f5d4491ee15d56faafc9677eef109e6951341acb54395f6d5" exitCode=0
Oct 31 01:14:22.615772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.614099    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs" event={"ID":"417b23a9-09ea-4427-8f72-85aad51e5fc6","Type":"ContainerDied","Data":"3df794e5fd30905f5d4491ee15d56faafc9677eef109e6951341acb54395f6d5"}
Oct 31 01:14:22.615772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.614145    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs" event={"ID":"417b23a9-09ea-4427-8f72-85aad51e5fc6","Type":"ContainerStarted","Data":"b3c88efc169b03da47b9933d288c240fa9f6da5ce41ac618fda68ffb6d578ac5"}
Oct 31 01:14:22.621674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.620422    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6" event={"ID":"0b67dbb9-2291-4645-9b71-926b6b936296","Type":"ContainerDied","Data":"44fd0cd7e2094c187c034cbe9b561940f0713a17886aaac5dba4a097abffd4ee"}
Oct 31 01:14:22.621674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.620460    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44fd0cd7e2094c187c034cbe9b561940f0713a17886aaac5dba4a097abffd4ee"
Oct 31 01:14:22.621674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.620506    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"
Oct 31 01:14:22.766134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.766031    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerName="payload-processing" containerID="cri-o://94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894" gracePeriod=25
Oct 31 01:14:22.950552 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.950520    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"]
Oct 31 01:14:22.958878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:22.958851    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-0a6cca-w9kvr"]
Oct 31 01:14:23.192715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.192464    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58689c65-3e8b-4e91-bdb6-5d4d71fec223" path="/var/lib/kubelet/pods/58689c65-3e8b-4e91-bdb6-5d4d71fec223/volumes"
Oct 31 01:14:23.849032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.848999    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:23.855075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.851279    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main" containerID="cri-o://57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a" gracePeriod=45
Oct 31 01:14:23.913084 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.912996    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv\" (UniqueName: \"kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv\") pod \"792d3fb7-b6f8-47fc-9017-e2a300023f16\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") "
Oct 31 01:14:23.913240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.913090    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca\") pod \"792d3fb7-b6f8-47fc-9017-e2a300023f16\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") "
Oct 31 01:14:23.913240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.913182    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca\") pod \"792d3fb7-b6f8-47fc-9017-e2a300023f16\" (UID: \"792d3fb7-b6f8-47fc-9017-e2a300023f16\") "
Oct 31 01:14:23.913842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.913767    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca" pod "792d3fb7-b6f8-47fc-9017-e2a300023f16" (UID: "792d3fb7-b6f8-47fc-9017-e2a300023f16"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:23.914125 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.913910    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:23.915453 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.915426    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca" pod "792d3fb7-b6f8-47fc-9017-e2a300023f16" (UID: "792d3fb7-b6f8-47fc-9017-e2a300023f16"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:23.915575 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.915484    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv" pod "792d3fb7-b6f8-47fc-9017-e2a300023f16" (UID: "792d3fb7-b6f8-47fc-9017-e2a300023f16"). InnerVolumeSpecName "kube-api-access-bw8nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:23.939269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:23.939244    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:24.017002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.015268    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca\") pod \"417b23a9-09ea-4427-8f72-85aad51e5fc6\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") "
Oct 31 01:14:24.017002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.015336    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca\") pod \"417b23a9-09ea-4427-8f72-85aad51e5fc6\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") "
Oct 31 01:14:24.017002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.015579    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd\" (UniqueName: \"kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd\") pod \"417b23a9-09ea-4427-8f72-85aad51e5fc6\" (UID: \"417b23a9-09ea-4427-8f72-85aad51e5fc6\") "
Oct 31 01:14:24.017326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.017120    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw8nv\" (UniqueName: \"kubernetes.io/projected/792d3fb7-b6f8-47fc-9017-e2a300023f16-kube-api-access-bw8nv\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.017326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.017150    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/792d3fb7-b6f8-47fc-9017-e2a300023f16-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.021442 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.018984    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca" pod "417b23a9-09ea-4427-8f72-85aad51e5fc6" (UID: "417b23a9-09ea-4427-8f72-85aad51e5fc6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:24.021442 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.020680    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca" pod "417b23a9-09ea-4427-8f72-85aad51e5fc6" (UID: "417b23a9-09ea-4427-8f72-85aad51e5fc6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:24.033842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.029231    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd" pod "417b23a9-09ea-4427-8f72-85aad51e5fc6" (UID: "417b23a9-09ea-4427-8f72-85aad51e5fc6"). InnerVolumeSpecName "kube-api-access-t4cgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:24.122514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.122480    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4cgd\" (UniqueName: \"kubernetes.io/projected/417b23a9-09ea-4427-8f72-85aad51e5fc6-kube-api-access-t4cgd\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.122514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.122515    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.122774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.122526    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/417b23a9-09ea-4427-8f72-85aad51e5fc6-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.218495 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.218470    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:14:24.325231 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325196    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2\" (UniqueName: \"kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2\") pod \"7132e0fc-e020-4ce9-9ca3-732da214abb6\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") "
Oct 31 01:14:24.325421 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325362    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home\") pod \"7132e0fc-e020-4ce9-9ca3-732da214abb6\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") "
Oct 31 01:14:24.325421 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325392    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache\") pod \"7132e0fc-e020-4ce9-9ca3-732da214abb6\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") "
Oct 31 01:14:24.325574 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325455    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir\") pod \"7132e0fc-e020-4ce9-9ca3-732da214abb6\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") "
Oct 31 01:14:24.325574 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325541    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm\") pod \"7132e0fc-e020-4ce9-9ca3-732da214abb6\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") "
Oct 31 01:14:24.325764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325595    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs\" (UniqueName: \"kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs\") pod \"7132e0fc-e020-4ce9-9ca3-732da214abb6\" (UID: \"7132e0fc-e020-4ce9-9ca3-732da214abb6\") "
Oct 31 01:14:24.325764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325669    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home" pod "7132e0fc-e020-4ce9-9ca3-732da214abb6" (UID: "7132e0fc-e020-4ce9-9ca3-732da214abb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:24.325764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325714    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache" pod "7132e0fc-e020-4ce9-9ca3-732da214abb6" (UID: "7132e0fc-e020-4ce9-9ca3-732da214abb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:24.325912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.325760    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir" pod "7132e0fc-e020-4ce9-9ca3-732da214abb6" (UID: "7132e0fc-e020-4ce9-9ca3-732da214abb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:24.326257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.326227    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.326257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.326253    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.326408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.326286    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.327975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.327935    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs" pod "7132e0fc-e020-4ce9-9ca3-732da214abb6" (UID: "7132e0fc-e020-4ce9-9ca3-732da214abb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:24.328129 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.328107    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm" pod "7132e0fc-e020-4ce9-9ca3-732da214abb6" (UID: "7132e0fc-e020-4ce9-9ca3-732da214abb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:24.328257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.328230    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2" pod "7132e0fc-e020-4ce9-9ca3-732da214abb6" (UID: "7132e0fc-e020-4ce9-9ca3-732da214abb6"). InnerVolumeSpecName "kube-api-access-fkst2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:24.427491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.427460    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7132e0fc-e020-4ce9-9ca3-732da214abb6-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.427491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.427491    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7132e0fc-e020-4ce9-9ca3-732da214abb6-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.427735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.427508    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkst2\" (UniqueName: \"kubernetes.io/projected/7132e0fc-e020-4ce9-9ca3-732da214abb6-kube-api-access-fkst2\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:24.659945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.656964    8029 generic.go:356] "Generic (PLEG): container finished" podID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerID="57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a" exitCode=0
Oct 31 01:14:24.659945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.657110    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" event={"ID":"7132e0fc-e020-4ce9-9ca3-732da214abb6","Type":"ContainerDied","Data":"57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a"}
Oct 31 01:14:24.659945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.657150    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp" event={"ID":"7132e0fc-e020-4ce9-9ca3-732da214abb6","Type":"ContainerDied","Data":"e54dbb333a1f4c4bedb532b33928127d8441c40dcf28022827b43c9e56edcd54"}
Oct 31 01:14:24.659945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.657172    8029 scope.go:122] "RemoveContainer" containerID="57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a"
Oct 31 01:14:24.659945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.657325    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"
Oct 31 01:14:24.673741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.665427    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"
Oct 31 01:14:24.673741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.666097    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt" event={"ID":"792d3fb7-b6f8-47fc-9017-e2a300023f16","Type":"ContainerDied","Data":"474c3b65e16ac67cd3306aaa78753199573c0ea99e6a1e2f06d7c36e3fc0ba5d"}
Oct 31 01:14:24.673741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.666154    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="474c3b65e16ac67cd3306aaa78753199573c0ea99e6a1e2f06d7c36e3fc0ba5d"
Oct 31 01:14:24.682976 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.682749    8029 scope.go:122] "RemoveContainer" containerID="57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a"
Oct 31 01:14:24.683089 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:24.683061    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a\": container with ID starting with 57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a not found: ID does not exist" containerID="57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a"
Oct 31 01:14:24.683146 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.683097    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a"} err="failed to get container status \"57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a\": rpc error: code = NotFound desc = could not find container \"57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a\": container with ID starting with 57089adb2f03f8950081ea8b968d4b6f0f6e7d4d8f5b6c5cb515fc0a13eb9b0a not found: ID does not exist"
Oct 31 01:14:24.683784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.683766    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"
Oct 31 01:14:24.684056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.684035    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs" event={"ID":"417b23a9-09ea-4427-8f72-85aad51e5fc6","Type":"ContainerDied","Data":"b3c88efc169b03da47b9933d288c240fa9f6da5ce41ac618fda68ffb6d578ac5"}
Oct 31 01:14:24.684097 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.684085    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3c88efc169b03da47b9933d288c240fa9f6da5ce41ac618fda68ffb6d578ac5"
Oct 31 01:14:24.707659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.707581    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"]
Oct 31 01:14:24.716373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:24.716341    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-8499c8/simulator-w4-8499c8-kserve-6cb6c79696-s9ddp"]
Oct 31 01:14:25.156151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.156122    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"]
Oct 31 01:14:25.159113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.159087    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:14:25.189788 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.189749    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" path="/var/lib/kubelet/pods/7132e0fc-e020-4ce9-9ca3-732da214abb6/volumes"
Oct 31 01:14:25.192644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.192618    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"]
Oct 31 01:14:25.239636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.239543    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.239807 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.239634    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.239807 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.239752    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.239807 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.239782    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.239945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.239931    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.240021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240009    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.240069 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240052    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.240109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240086    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.240154 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240142    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data\") pod \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\" (UID: \"d6c899ad-9c8f-478e-a1f9-101d57b4098f\") "
Oct 31 01:14:25.240645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240615    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:25.240757 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240727    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.241279 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.240991    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.241279 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.241018    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.241279 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.241235    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.241279 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.241254    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.241835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.241813    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.244237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.243652    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.244237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.243670    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "kube-api-access-wqnm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.244468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.244443    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.245494 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.245405    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo" pod "d6c899ad-9c8f-478e-a1f9-101d57b4098f" (UID: "d6c899ad-9c8f-478e-a1f9-101d57b4098f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:25.315310 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.315279    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"]
Oct 31 01:14:25.342993 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.342961    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.342993 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.342992    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.343201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.343005    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.343201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.343019    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqnm9\" (UniqueName: \"kubernetes.io/projected/d6c899ad-9c8f-478e-a1f9-101d57b4098f-kube-api-access-wqnm9\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.343201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.343029    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.343201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.343038    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.343201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.343046    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c899ad-9c8f-478e-a1f9-101d57b4098f-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.521960    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"]
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.522983    8029 cpu_manager.go:414] "containerMap: removing container" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523000    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523017    8029 cpu_manager.go:414] "containerMap: removing container" podUID="417b23a9-09ea-4427-8f72-85aad51e5fc6" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523022    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="417b23a9-09ea-4427-8f72-85aad51e5fc6" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523035    8029 cpu_manager.go:414] "containerMap: removing container" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523040    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523049    8029 cpu_manager.go:414] "containerMap: removing container" podUID="0b67dbb9-2291-4645-9b71-926b6b936296" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523054    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b67dbb9-2291-4645-9b71-926b6b936296" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523092    8029 cpu_manager.go:414] "containerMap: removing container" podUID="792d3fb7-b6f8-47fc-9017-e2a300023f16" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523097    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="792d3fb7-b6f8-47fc-9017-e2a300023f16" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.523203    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="792d3fb7-b6f8-47fc-9017-e2a300023f16" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.524504    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerName="istio-proxy"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.524533    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="417b23a9-09ea-4427-8f72-85aad51e5fc6" containerName="revoke-keys"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.524544    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7132e0fc-e020-4ce9-9ca3-732da214abb6" containerName="main"
Oct 31 01:14:25.527303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.524555    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b67dbb9-2291-4645-9b71-926b6b936296" containerName="revoke-keys"
Oct 31 01:14:25.555027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.551744    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"]
Oct 31 01:14:25.555027 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.551903    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.556283 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.556264    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:14:25.556499 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.556488    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:14:25.560750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.558130    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:14:25.560750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.559999    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:14:25.570717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.570691    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"]
Oct 31 01:14:25.580964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.580942    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.592030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.591998    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"]
Oct 31 01:14:25.650875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.650846    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.651138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.651121    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgdzd\" (UniqueName: \"kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.651441 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.651425    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.651572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.651560    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfxfk\" (UniqueName: \"kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.651785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.651693    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.651922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.651907    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.715775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.715727    8029 generic.go:356] "Generic (PLEG): container finished" podID="87692b05-4276-4081-a63c-60cd3489b10b" containerID="5184b2c36b80dc5962ef2a49129e7bc8de84845e483e141b6335d2bc51931ac1" exitCode=0
Oct 31 01:14:25.716017 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.715984    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" event={"ID":"87692b05-4276-4081-a63c-60cd3489b10b","Type":"ContainerDied","Data":"5184b2c36b80dc5962ef2a49129e7bc8de84845e483e141b6335d2bc51931ac1"}
Oct 31 01:14:25.719041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.719011    8029 generic.go:356] "Generic (PLEG): container finished" podID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerID="a6b5accd13235ac4c2ec1f02c72ae82e678fbfa388120ad11c88cc01cc64fb6d" exitCode=0
Oct 31 01:14:25.719148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.719108    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" event={"ID":"7cf65f8b-7463-4c96-87ab-80e5d279210d","Type":"ContainerDied","Data":"a6b5accd13235ac4c2ec1f02c72ae82e678fbfa388120ad11c88cc01cc64fb6d"}
Oct 31 01:14:25.721314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.721292    8029 generic.go:356] "Generic (PLEG): container finished" podID="a043128b-3095-44ca-877b-173faa3a2099" containerID="d75dde75e6f4dfd8bfcabcb0e88cd9e1714d1a2a0d795c07e2dde57333202c56" exitCode=0
Oct 31 01:14:25.721591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.721575    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" event={"ID":"a043128b-3095-44ca-877b-173faa3a2099","Type":"ContainerDied","Data":"d75dde75e6f4dfd8bfcabcb0e88cd9e1714d1a2a0d795c07e2dde57333202c56"}
Oct 31 01:14:25.723565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.723543    8029 generic.go:356] "Generic (PLEG): container finished" podID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" containerID="bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c" exitCode=0
Oct 31 01:14:25.723728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.723659    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" event={"ID":"d6c899ad-9c8f-478e-a1f9-101d57b4098f","Type":"ContainerDied","Data":"bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c"}
Oct 31 01:14:25.723728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.723703    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv" event={"ID":"d6c899ad-9c8f-478e-a1f9-101d57b4098f","Type":"ContainerDied","Data":"3dcb92a0c9acc4e56d0282a01b33c2821d56dc10c0d72bd9f6bc18ed33022840"}
Oct 31 01:14:25.723851 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.723758    8029 scope.go:122] "RemoveContainer" containerID="bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c"
Oct 31 01:14:25.723941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.723925    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"
Oct 31 01:14:25.746397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.745758    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:14:25.750268 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.750241    8029 scope.go:122] "RemoveContainer" containerID="bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c"
Oct 31 01:14:25.750559 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:25.750539    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c\": container with ID starting with bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c not found: ID does not exist" containerID="bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c"
Oct 31 01:14:25.750703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.750568    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c"} err="failed to get container status \"bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c\": rpc error: code = NotFound desc = could not find container \"bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c\": container with ID starting with bced6a696ce896835525fec9ae1b1acb76d80d456879770986d444660af8339c not found: ID does not exist"
Oct 31 01:14:25.755643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.755569    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.756096 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.756066    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgdzd\" (UniqueName: \"kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.756354 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.756331    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.756431 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.756409    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfxfk\" (UniqueName: \"kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.756488 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.756457    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.756542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.756499    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.761809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.761785    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.766202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.762328    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.766202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.762457    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.766202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.765316    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.772014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.770135    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfxfk\" (UniqueName: \"kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk\") pod \"maas-api-revoke-sub-73974aa0-2562q\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.772014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.770135    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgdzd\" (UniqueName: \"kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd\") pod \"maas-api-revoke-sub-4f95bdf2-d7lnk\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.775800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.775308    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"]
Oct 31 01:14:25.781525 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.781444    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:14:25.784436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.784414    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-swkbv"]
Oct 31 01:14:25.852467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.852437    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:14:25.857553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.857526    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.857739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.857638    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.857848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.857817    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.857963 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.857897    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.857964    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:25.858089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858033    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858084    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858117    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858272    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858293    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:25.858518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858368    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858423    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858453    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858513    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858562    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858638    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858741    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket\") pod \"87692b05-4276-4081-a63c-60cd3489b10b\" (UID: \"87692b05-4276-4081-a63c-60cd3489b10b\") "
Oct 31 01:14:25.858970 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858822    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858970 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858894    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.858970 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.858950    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token\") pod \"7cf65f8b-7463-4c96-87ab-80e5d279210d\" (UID: \"7cf65f8b-7463-4c96-87ab-80e5d279210d\") "
Oct 31 01:14:25.859186 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.859160    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.859376 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.859353    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.859892    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.859920    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.859934    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87692b05-4276-4081-a63c-60cd3489b10b-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.859949    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cf65f8b-7463-4c96-87ab-80e5d279210d-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.860268    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.860447    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.860466    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.860709    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.860919    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.861549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.861266    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.862124 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.862027    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.862458 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.862431    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.863396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.863364    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:25.863876 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.863852    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.863876 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.863853    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq" pod "7cf65f8b-7463-4c96-87ab-80e5d279210d" (UID: "7cf65f8b-7463-4c96-87ab-80e5d279210d"). InnerVolumeSpecName "kube-api-access-5vzpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.864561 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.864536    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:25.864790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.864768    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.865375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.865353    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd" pod "87692b05-4276-4081-a63c-60cd3489b10b" (UID: "87692b05-4276-4081-a63c-60cd3489b10b"). InnerVolumeSpecName "kube-api-access-m9cjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.888129 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.888100    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:25.902519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.901799    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:25.961670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961321    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.961670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961410    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.961670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961494    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.961670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961534    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.961670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961595    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961680    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961708    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961795    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.961918    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy\") pod \"a043128b-3095-44ca-877b-173faa3a2099\" (UID: \"a043128b-3095-44ca-877b-173faa3a2099\") "
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962717    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87692b05-4276-4081-a63c-60cd3489b10b-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962740    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962754    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962768    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vzpq\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-kube-api-access-5vzpq\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962782    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962795    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962811    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9cjd\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-kube-api-access-m9cjd\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962826    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962839    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962852    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962864    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cf65f8b-7463-4c96-87ab-80e5d279210d-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962878    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cf65f8b-7463-4c96-87ab-80e5d279210d-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962891    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87692b05-4276-4081-a63c-60cd3489b10b-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.963975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.962905    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87692b05-4276-4081-a63c-60cd3489b10b-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:25.967296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.964768    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.967361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.967336    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.970405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.967732    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:25.970405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.969193    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.970405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.969997    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:25.970405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.970148    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:25.970405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.970261    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.970405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.970341    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:25.972572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:25.972540    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v" pod "a043128b-3095-44ca-877b-173faa3a2099" (UID: "a043128b-3095-44ca-877b-173faa3a2099"). InnerVolumeSpecName "kube-api-access-g2c2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064750    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064788    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064803    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064816    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064830    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2c2v\" (UniqueName: \"kubernetes.io/projected/a043128b-3095-44ca-877b-173faa3a2099-kube-api-access-g2c2v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064846    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a043128b-3095-44ca-877b-173faa3a2099-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.064860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064861    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.065572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064875    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a043128b-3095-44ca-877b-173faa3a2099-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.065572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.064888    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a043128b-3095-44ca-877b-173faa3a2099-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:26.198163 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:26.198127    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod510bd72f_cad7_4cb0_a550_5fc9031ef218.slice/crio-ed80f2c9f255ca32bb5b0f4861c022001428099d311fabb08ebbf74676c09d34 WatchSource:0}: Error finding container ed80f2c9f255ca32bb5b0f4861c022001428099d311fabb08ebbf74676c09d34: Status 404 returned error can't find the container with id ed80f2c9f255ca32bb5b0f4861c022001428099d311fabb08ebbf74676c09d34
Oct 31 01:14:26.199187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.199162    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"]
Oct 31 01:14:26.212198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.212162    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"]
Oct 31 01:14:26.215230 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:26.215201    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063a2658_5310_4b07_bbe6_b87b7d875940.slice/crio-96456b4c1257ed1ba9c8dbac00e40b432c9c09476d3b8890a5ad7cf37d45dbda WatchSource:0}: Error finding container 96456b4c1257ed1ba9c8dbac00e40b432c9c09476d3b8890a5ad7cf37d45dbda: Status 404 returned error can't find the container with id 96456b4c1257ed1ba9c8dbac00e40b432c9c09476d3b8890a5ad7cf37d45dbda
Oct 31 01:14:26.733647 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.733513    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q" event={"ID":"510bd72f-cad7-4cb0-a550-5fc9031ef218","Type":"ContainerStarted","Data":"f86b23bf04e9b56b970f9beb9c572b6c61dd7fd5da91ac3ddbdd897717d31179"}
Oct 31 01:14:26.733647 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.733575    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q" event={"ID":"510bd72f-cad7-4cb0-a550-5fc9031ef218","Type":"ContainerStarted","Data":"ed80f2c9f255ca32bb5b0f4861c022001428099d311fabb08ebbf74676c09d34"}
Oct 31 01:14:26.736349 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.736322    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk" event={"ID":"063a2658-5310-4b07-bbe6-b87b7d875940","Type":"ContainerStarted","Data":"b71c5db2c4961d1d18b6166323ef460321c91422b9fe7e9b2969f6215b121916"}
Oct 31 01:14:26.736457 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.736371    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk" event={"ID":"063a2658-5310-4b07-bbe6-b87b7d875940","Type":"ContainerStarted","Data":"96456b4c1257ed1ba9c8dbac00e40b432c9c09476d3b8890a5ad7cf37d45dbda"}
Oct 31 01:14:26.737713 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.737694    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"
Oct 31 01:14:26.737789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.737729    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh" event={"ID":"87692b05-4276-4081-a63c-60cd3489b10b","Type":"ContainerDied","Data":"9f17aa2c71b30560ec0bf8e0fd0142057792aaa6b7fcca9965d16c471fae7fa7"}
Oct 31 01:14:26.737789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.737773    8029 scope.go:122] "RemoveContainer" containerID="5184b2c36b80dc5962ef2a49129e7bc8de84845e483e141b6335d2bc51931ac1"
Oct 31 01:14:26.739451 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.739422    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"
Oct 31 01:14:26.739547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.739452    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4" event={"ID":"7cf65f8b-7463-4c96-87ab-80e5d279210d","Type":"ContainerDied","Data":"f3764d0b0de010120cec4b96c975a9ecb988a2c8ee01c19fa24c15ae2decec5d"}
Oct 31 01:14:26.741403 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.741378    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"
Oct 31 01:14:26.741498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.741402    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g" event={"ID":"a043128b-3095-44ca-877b-173faa3a2099","Type":"ContainerDied","Data":"c6ebecb0b03dffcab62869681d6c5f2939fc6ff4fa23c3010d0ea19789fc42f9"}
Oct 31 01:14:26.753041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.752985    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q" podStartSLOduration=1.752968235 podStartE2EDuration="1.752968235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:26.752315252 +0000 UTC m=+1063.739427233" watchObservedRunningTime="2026-10-31 01:14:26.752968235 +0000 UTC m=+1063.740080212"
Oct 31 01:14:26.754061 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.754040    8029 scope.go:122] "RemoveContainer" containerID="a6b5accd13235ac4c2ec1f02c72ae82e678fbfa388120ad11c88cc01cc64fb6d"
Oct 31 01:14:26.772486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.772465    8029 scope.go:122] "RemoveContainer" containerID="d75dde75e6f4dfd8bfcabcb0e88cd9e1714d1a2a0d795c07e2dde57333202c56"
Oct 31 01:14:26.783215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.783173    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk" podStartSLOduration=1.78316012 podStartE2EDuration="1.78316012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:26.782272048 +0000 UTC m=+1063.769384025" watchObservedRunningTime="2026-10-31 01:14:26.78316012 +0000 UTC m=+1063.770272102"
Oct 31 01:14:26.801358 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.801336    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"]
Oct 31 01:14:26.807204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.807183    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-bz2d4"]
Oct 31 01:14:26.822676 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.821023    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"]
Oct 31 01:14:26.850161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.850111    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-dvf8g"]
Oct 31 01:14:26.859034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.859003    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"]
Oct 31 01:14:26.865208 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:26.865179    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1b8d05-openshift-default-659bf8c59c-df2lh"]
Oct 31 01:14:27.185584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.185553    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" path="/var/lib/kubelet/pods/7cf65f8b-7463-4c96-87ab-80e5d279210d/volumes"
Oct 31 01:14:27.186203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.186180    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87692b05-4276-4081-a63c-60cd3489b10b" path="/var/lib/kubelet/pods/87692b05-4276-4081-a63c-60cd3489b10b/volumes"
Oct 31 01:14:27.186718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.186702    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a043128b-3095-44ca-877b-173faa3a2099" path="/var/lib/kubelet/pods/a043128b-3095-44ca-877b-173faa3a2099/volumes"
Oct 31 01:14:27.187493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.187473    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6c899ad-9c8f-478e-a1f9-101d57b4098f" path="/var/lib/kubelet/pods/d6c899ad-9c8f-478e-a1f9-101d57b4098f/volumes"
Oct 31 01:14:27.754437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.754400    8029 generic.go:356] "Generic (PLEG): container finished" podID="510bd72f-cad7-4cb0-a550-5fc9031ef218" containerID="f86b23bf04e9b56b970f9beb9c572b6c61dd7fd5da91ac3ddbdd897717d31179" exitCode=0
Oct 31 01:14:27.754870 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.754491    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q" event={"ID":"510bd72f-cad7-4cb0-a550-5fc9031ef218","Type":"ContainerDied","Data":"f86b23bf04e9b56b970f9beb9c572b6c61dd7fd5da91ac3ddbdd897717d31179"}
Oct 31 01:14:27.756106 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.756087    8029 generic.go:356] "Generic (PLEG): container finished" podID="063a2658-5310-4b07-bbe6-b87b7d875940" containerID="b71c5db2c4961d1d18b6166323ef460321c91422b9fe7e9b2969f6215b121916" exitCode=0
Oct 31 01:14:27.756183 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:27.756121    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk" event={"ID":"063a2658-5310-4b07-bbe6-b87b7d875940","Type":"ContainerDied","Data":"b71c5db2c4961d1d18b6166323ef460321c91422b9fe7e9b2969f6215b121916"}
Oct 31 01:14:28.977966 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:28.977942    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:28.980732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:28.980711    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:29.109503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109471    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca\") pod \"063a2658-5310-4b07-bbe6-b87b7d875940\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") "
Oct 31 01:14:29.109701 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109523    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk\" (UniqueName: \"kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk\") pod \"510bd72f-cad7-4cb0-a550-5fc9031ef218\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") "
Oct 31 01:14:29.109701 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109582    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca\") pod \"510bd72f-cad7-4cb0-a550-5fc9031ef218\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") "
Oct 31 01:14:29.109820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109738    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca\") pod \"063a2658-5310-4b07-bbe6-b87b7d875940\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") "
Oct 31 01:14:29.109901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109867    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca\") pod \"510bd72f-cad7-4cb0-a550-5fc9031ef218\" (UID: \"510bd72f-cad7-4cb0-a550-5fc9031ef218\") "
Oct 31 01:14:29.110017 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109997    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd\" (UniqueName: \"kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd\") pod \"063a2658-5310-4b07-bbe6-b87b7d875940\" (UID: \"063a2658-5310-4b07-bbe6-b87b7d875940\") "
Oct 31 01:14:29.110017 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.109998    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca" pod "510bd72f-cad7-4cb0-a550-5fc9031ef218" (UID: "510bd72f-cad7-4cb0-a550-5fc9031ef218"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:29.110395 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.110048    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca" pod "063a2658-5310-4b07-bbe6-b87b7d875940" (UID: "063a2658-5310-4b07-bbe6-b87b7d875940"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:29.110891 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.110868    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:29.111009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.110896    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:29.112298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.112255    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk" pod "510bd72f-cad7-4cb0-a550-5fc9031ef218" (UID: "510bd72f-cad7-4cb0-a550-5fc9031ef218"). InnerVolumeSpecName "kube-api-access-wfxfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:29.112426 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.112406    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca" pod "063a2658-5310-4b07-bbe6-b87b7d875940" (UID: "063a2658-5310-4b07-bbe6-b87b7d875940"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:29.112589 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.112568    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca" pod "510bd72f-cad7-4cb0-a550-5fc9031ef218" (UID: "510bd72f-cad7-4cb0-a550-5fc9031ef218"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:29.112770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.112751    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd" pod "063a2658-5310-4b07-bbe6-b87b7d875940" (UID: "063a2658-5310-4b07-bbe6-b87b7d875940"). InnerVolumeSpecName "kube-api-access-tgdzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:29.212563 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.212538    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgdzd\" (UniqueName: \"kubernetes.io/projected/063a2658-5310-4b07-bbe6-b87b7d875940-kube-api-access-tgdzd\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:29.212563 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.212563    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/063a2658-5310-4b07-bbe6-b87b7d875940-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:29.212764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.212573    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfxfk\" (UniqueName: \"kubernetes.io/projected/510bd72f-cad7-4cb0-a550-5fc9031ef218-kube-api-access-wfxfk\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:29.212764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.212583    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/510bd72f-cad7-4cb0-a550-5fc9031ef218-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:29.561307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.561213    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:29.725031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.724990    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"]
Oct 31 01:14:29.725297 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.725269    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy" containerID="cri-o://6d602f1615bfaf5502bd86c375f265d4c4f1eb5a817bd74bccf52042a1730cfd" gracePeriod=30
Oct 31 01:14:29.735711 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.735678    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"]
Oct 31 01:14:29.735943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.735904    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy" containerID="cri-o://2701c160c1b02c9a3b1ead6165e7f18a40200af45c2bc316b8b020da3ceec1b7" gracePeriod=30
Oct 31 01:14:29.741701 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.741676    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"]
Oct 31 01:14:29.744686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.741860    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" containerID="cri-o://c1a18ecba2b13d0e00478b5bea8412b8a86c2f98368752b722e50f3961b505bf" gracePeriod=30
Oct 31 01:14:29.775967 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.775937    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"
Oct 31 01:14:29.775967 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.775948    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q" event={"ID":"510bd72f-cad7-4cb0-a550-5fc9031ef218","Type":"ContainerDied","Data":"ed80f2c9f255ca32bb5b0f4861c022001428099d311fabb08ebbf74676c09d34"}
Oct 31 01:14:29.776194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.776008    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed80f2c9f255ca32bb5b0f4861c022001428099d311fabb08ebbf74676c09d34"
Oct 31 01:14:29.777531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.777504    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"
Oct 31 01:14:29.781951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.777527    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk" event={"ID":"063a2658-5310-4b07-bbe6-b87b7d875940","Type":"ContainerDied","Data":"96456b4c1257ed1ba9c8dbac00e40b432c9c09476d3b8890a5ad7cf37d45dbda"}
Oct 31 01:14:29.781951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:29.777570    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96456b4c1257ed1ba9c8dbac00e40b432c9c09476d3b8890a5ad7cf37d45dbda"
Oct 31 01:14:30.403514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:30.403474    8029 patch_prober.go:29] interesting pod/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 31 01:14:30.404058 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:30.403541    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:30.567190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:30.567155    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:30.695328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:30.695251    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"]
Oct 31 01:14:30.727059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:30.726533    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"]
Oct 31 01:14:30.920029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:30.919988    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"]
Oct 31 01:14:32.392280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.392239    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"]
Oct 31 01:14:32.393175 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393139    8029 cpu_manager.go:414] "containerMap: removing container" podUID="510bd72f-cad7-4cb0-a550-5fc9031ef218" containerName="revoke-keys"
Oct 31 01:14:32.393175 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393163    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="510bd72f-cad7-4cb0-a550-5fc9031ef218" containerName="revoke-keys"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393189    8029 cpu_manager.go:414] "containerMap: removing container" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393196    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393206    8029 cpu_manager.go:414] "containerMap: removing container" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393213    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393254    8029 cpu_manager.go:414] "containerMap: removing container" podUID="063a2658-5310-4b07-bbe6-b87b7d875940" containerName="revoke-keys"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393261    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="063a2658-5310-4b07-bbe6-b87b7d875940" containerName="revoke-keys"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393300    8029 cpu_manager.go:414] "containerMap: removing container" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy"
Oct 31 01:14:32.393338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393308    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy"
Oct 31 01:14:32.393756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393484    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="063a2658-5310-4b07-bbe6-b87b7d875940" containerName="revoke-keys"
Oct 31 01:14:32.393756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393505    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7cf65f8b-7463-4c96-87ab-80e5d279210d" containerName="istio-proxy"
Oct 31 01:14:32.393756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393518    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="510bd72f-cad7-4cb0-a550-5fc9031ef218" containerName="revoke-keys"
Oct 31 01:14:32.393756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393529    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="87692b05-4276-4081-a63c-60cd3489b10b" containerName="istio-proxy"
Oct 31 01:14:32.393756 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.393538    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a043128b-3095-44ca-877b-173faa3a2099" containerName="istio-proxy"
Oct 31 01:14:32.399218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.399194    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.402750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.402725    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:14:32.402891 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.402767    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"]
Oct 31 01:14:32.403659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.403340    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:14:32.403659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.403367    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:14:32.403659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.403455    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:14:32.407912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.407892    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.409569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.409359    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"]
Oct 31 01:14:32.425377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.424873    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"]
Oct 31 01:14:32.438468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.438441    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"]
Oct 31 01:14:32.443933 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.443900    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.453449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.453420    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"]
Oct 31 01:14:32.520006 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.519963    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxc8v\" (UniqueName: \"kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.520162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.520028    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.520162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.520113    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.520162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.520144    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8x5\" (UniqueName: \"kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.520291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.520259    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.520455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.520442    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.622026 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.621988    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.622026 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622026    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxc8v\" (UniqueName: \"kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.622254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622050    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.622254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622086    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.622254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622120    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8x5\" (UniqueName: \"kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.622254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622233    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.622479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622286    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.622479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622316    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtw7q\" (UniqueName: \"kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.622479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622368    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.622914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622887    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.623044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.622933    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.624692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.624666    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.624789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.624674    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.630179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.630155    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxc8v\" (UniqueName: \"kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v\") pod \"maas-api-revoke-sub-10e06873-q9628\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.630293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.630224    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8x5\" (UniqueName: \"kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5\") pod \"maas-api-revoke-sub-cec76339-j97kz\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.724039 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.723945    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:32.724462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.724435    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.724576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.724479    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtw7q\" (UniqueName: \"kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.724576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.724527    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.725247 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.725220    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.727146 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.727125    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.732127 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.732104    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtw7q\" (UniqueName: \"kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q\") pod \"maas-api-revoke-sub-78e20883-26dxz\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:32.734366 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.734345    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:32.764423 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:32.764390    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:33.020140 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:33.020110    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743abb0c_6a7d_428d_9fae_c153f886345b.slice/crio-c95b27a9230c447f41a3158cd7097d96087e3b28fff3c2796961ab2b51da5885 WatchSource:0}: Error finding container c95b27a9230c447f41a3158cd7097d96087e3b28fff3c2796961ab2b51da5885: Status 404 returned error can't find the container with id c95b27a9230c447f41a3158cd7097d96087e3b28fff3c2796961ab2b51da5885
Oct 31 01:14:33.022774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.022753    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"]
Oct 31 01:14:33.066685 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:33.066658    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3feb32fa_9d2d_4d52_b778_40e57217bbfa.slice/crio-3c1ee131c645484214f6f2074d7e42853c3b047f39a003c17c3ac0e51e91d741 WatchSource:0}: Error finding container 3c1ee131c645484214f6f2074d7e42853c3b047f39a003c17c3ac0e51e91d741: Status 404 returned error can't find the container with id 3c1ee131c645484214f6f2074d7e42853c3b047f39a003c17c3ac0e51e91d741
Oct 31 01:14:33.068870 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.068846    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"]
Oct 31 01:14:33.150368 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:33.150335    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c601844_fa2c_4865_befd_03f0dc788725.slice/crio-99c379ca0bbe11ad261cdf6665ee2cee1d5b96c3fdb465693ccfcd2684e0ba21 WatchSource:0}: Error finding container 99c379ca0bbe11ad261cdf6665ee2cee1d5b96c3fdb465693ccfcd2684e0ba21: Status 404 returned error can't find the container with id 99c379ca0bbe11ad261cdf6665ee2cee1d5b96c3fdb465693ccfcd2684e0ba21
Oct 31 01:14:33.151365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.151343    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"]
Oct 31 01:14:33.734280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.734248    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:14:33.755077 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.754092    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume\") pod \"448f6a3f-07c9-4381-a356-ec466a740f46\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") "
Oct 31 01:14:33.755077 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.754269    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj\" (UniqueName: \"kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj\") pod \"448f6a3f-07c9-4381-a356-ec466a740f46\" (UID: \"448f6a3f-07c9-4381-a356-ec466a740f46\") "
Oct 31 01:14:33.755077 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.754464    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume" pod "448f6a3f-07c9-4381-a356-ec466a740f46" (UID: "448f6a3f-07c9-4381-a356-ec466a740f46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:33.755077 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.755040    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/448f6a3f-07c9-4381-a356-ec466a740f46-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:33.756764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.756735    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj" pod "448f6a3f-07c9-4381-a356-ec466a740f46" (UID: "448f6a3f-07c9-4381-a356-ec466a740f46"). InnerVolumeSpecName "kube-api-access-p65cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:33.857273 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.857239    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p65cj\" (UniqueName: \"kubernetes.io/projected/448f6a3f-07c9-4381-a356-ec466a740f46-kube-api-access-p65cj\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:33.901028 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.900994    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz" event={"ID":"3c601844-fa2c-4865-befd-03f0dc788725","Type":"ContainerStarted","Data":"7c184dd284fbb5eb1a5d006ae2216c46d00e354acc164dd4bd279ad921c7f2b9"}
Oct 31 01:14:33.901198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.901052    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz" event={"ID":"3c601844-fa2c-4865-befd-03f0dc788725","Type":"ContainerStarted","Data":"99c379ca0bbe11ad261cdf6665ee2cee1d5b96c3fdb465693ccfcd2684e0ba21"}
Oct 31 01:14:33.902497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.902472    8029 generic.go:356] "Generic (PLEG): container finished" podID="448f6a3f-07c9-4381-a356-ec466a740f46" containerID="eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb" exitCode=0
Oct 31 01:14:33.902665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.902507    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"
Oct 31 01:14:33.902665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.902565    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" event={"ID":"448f6a3f-07c9-4381-a356-ec466a740f46","Type":"ContainerDied","Data":"eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb"}
Oct 31 01:14:33.902665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.902628    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2" event={"ID":"448f6a3f-07c9-4381-a356-ec466a740f46","Type":"ContainerDied","Data":"8d35f355e98b149df91d01678df1d2aef0eb1a4217487905adc97e7c6515d154"}
Oct 31 01:14:33.902665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.902659    8029 scope.go:122] "RemoveContainer" containerID="eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb"
Oct 31 01:14:33.904690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.904664    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628" event={"ID":"743abb0c-6a7d-428d-9fae-c153f886345b","Type":"ContainerStarted","Data":"deadab4dcfdbfdbe010ada28d626878b5c4d9c9aa2618a87d782819672fa259f"}
Oct 31 01:14:33.904797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.904712    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628" event={"ID":"743abb0c-6a7d-428d-9fae-c153f886345b","Type":"ContainerStarted","Data":"c95b27a9230c447f41a3158cd7097d96087e3b28fff3c2796961ab2b51da5885"}
Oct 31 01:14:33.910065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.910044    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz" event={"ID":"3feb32fa-9d2d-4d52-b778-40e57217bbfa","Type":"ContainerStarted","Data":"f50eb25b879a59fd6f8d514cfa6e8ed2c2edd8d59ee40e62977ec42de95c10eb"}
Oct 31 01:14:33.910183 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.910080    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz" event={"ID":"3feb32fa-9d2d-4d52-b778-40e57217bbfa","Type":"ContainerStarted","Data":"3c1ee131c645484214f6f2074d7e42853c3b047f39a003c17c3ac0e51e91d741"}
Oct 31 01:14:33.915067 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.915052    8029 scope.go:122] "RemoveContainer" containerID="eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb"
Oct 31 01:14:33.915322 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:33.915305    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb\": container with ID starting with eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb not found: ID does not exist" containerID="eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb"
Oct 31 01:14:33.915394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.915329    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb"} err="failed to get container status \"eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb\": rpc error: code = NotFound desc = could not find container \"eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb\": container with ID starting with eb5e71cf16932ae5a13f27433ea268d0d3f4d18712b5682e0231d3c2d4e89dcb not found: ID does not exist"
Oct 31 01:14:33.921210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.921171    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz" podStartSLOduration=1.92115857 podStartE2EDuration="1.92115857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:33.920466339 +0000 UTC m=+1070.907578318" watchObservedRunningTime="2026-10-31 01:14:33.92115857 +0000 UTC m=+1070.908270562"
Oct 31 01:14:33.939589 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.939564    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"]
Oct 31 01:14:33.944667 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.944649    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0a6cca-769c8d94ff-9r6x2"]
Oct 31 01:14:33.954538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.954498    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628" podStartSLOduration=1.954484984 podStartE2EDuration="1.954484984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:33.949789643 +0000 UTC m=+1070.936901619" watchObservedRunningTime="2026-10-31 01:14:33.954484984 +0000 UTC m=+1070.941596950"
Oct 31 01:14:33.967895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:33.967851    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz" podStartSLOduration=1.9678384389999999 podStartE2EDuration="1.967838439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:14:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:14:33.965054558 +0000 UTC m=+1070.952166536" watchObservedRunningTime="2026-10-31 01:14:33.967838439 +0000 UTC m=+1070.954950415"
Oct 31 01:14:34.923698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.923654    8029 generic.go:356] "Generic (PLEG): container finished" podID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerID="2701c160c1b02c9a3b1ead6165e7f18a40200af45c2bc316b8b020da3ceec1b7" exitCode=0
Oct 31 01:14:34.926227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.923771    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" event={"ID":"21bfd530-3cf3-4784-81f1-d78f9fb72192","Type":"ContainerDied","Data":"2701c160c1b02c9a3b1ead6165e7f18a40200af45c2bc316b8b020da3ceec1b7"}
Oct 31 01:14:34.928872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.928843    8029 generic.go:356] "Generic (PLEG): container finished" podID="3c601844-fa2c-4865-befd-03f0dc788725" containerID="7c184dd284fbb5eb1a5d006ae2216c46d00e354acc164dd4bd279ad921c7f2b9" exitCode=0
Oct 31 01:14:34.929471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.929442    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz" event={"ID":"3c601844-fa2c-4865-befd-03f0dc788725","Type":"ContainerDied","Data":"7c184dd284fbb5eb1a5d006ae2216c46d00e354acc164dd4bd279ad921c7f2b9"}
Oct 31 01:14:34.937552 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.937515    8029 generic.go:356] "Generic (PLEG): container finished" podID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerID="c1a18ecba2b13d0e00478b5bea8412b8a86c2f98368752b722e50f3961b505bf" exitCode=0
Oct 31 01:14:34.937690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.937665    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" event={"ID":"fda6944e-5abf-4cbb-a38d-1ebc269b6a05","Type":"ContainerDied","Data":"c1a18ecba2b13d0e00478b5bea8412b8a86c2f98368752b722e50f3961b505bf"}
Oct 31 01:14:34.939664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.939639    8029 generic.go:356] "Generic (PLEG): container finished" podID="743abb0c-6a7d-428d-9fae-c153f886345b" containerID="deadab4dcfdbfdbe010ada28d626878b5c4d9c9aa2618a87d782819672fa259f" exitCode=0
Oct 31 01:14:34.939821 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.939799    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628" event={"ID":"743abb0c-6a7d-428d-9fae-c153f886345b","Type":"ContainerDied","Data":"deadab4dcfdbfdbe010ada28d626878b5c4d9c9aa2618a87d782819672fa259f"}
Oct 31 01:14:34.942297 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.942276    8029 generic.go:356] "Generic (PLEG): container finished" podID="e08a8410-136a-4178-80ce-dd599ca42048" containerID="6d602f1615bfaf5502bd86c375f265d4c4f1eb5a817bd74bccf52042a1730cfd" exitCode=0
Oct 31 01:14:34.942416 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.942358    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" event={"ID":"e08a8410-136a-4178-80ce-dd599ca42048","Type":"ContainerDied","Data":"6d602f1615bfaf5502bd86c375f265d4c4f1eb5a817bd74bccf52042a1730cfd"}
Oct 31 01:14:34.942464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.942422    8029 scope.go:122] "RemoveContainer" containerID="77d37e1c01078071eb108b000cbcd0dfac1eaa448c894cef2a5357d38e8adcb2"
Oct 31 01:14:34.944832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.944644    8029 generic.go:356] "Generic (PLEG): container finished" podID="3feb32fa-9d2d-4d52-b778-40e57217bbfa" containerID="f50eb25b879a59fd6f8d514cfa6e8ed2c2edd8d59ee40e62977ec42de95c10eb" exitCode=0
Oct 31 01:14:34.944832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:34.944777    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz" event={"ID":"3feb32fa-9d2d-4d52-b778-40e57217bbfa","Type":"ContainerDied","Data":"f50eb25b879a59fd6f8d514cfa6e8ed2c2edd8d59ee40e62977ec42de95c10eb"}
Oct 31 01:14:35.037697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.037570    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:14:35.075776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.075749    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.075969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.075881    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.075969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.075940    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.076090 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076060    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.076144 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076110    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.076194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076145    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.076194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076179    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.076296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076235    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.076345 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076326    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo\") pod \"21bfd530-3cf3-4784-81f1-d78f9fb72192\" (UID: \"21bfd530-3cf3-4784-81f1-d78f9fb72192\") "
Oct 31 01:14:35.077093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.076860    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:35.077093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.077034    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.077475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.077238    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.077475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.077362    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.077774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.077579    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.078190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.078163    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bfd530-3cf3-4784-81f1-d78f9fb72192-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.078278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.078196    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.078278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.078212    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.078278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.078226    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.078278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.078239    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.079467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.079434    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:35.080072 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.080053    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "kube-api-access-qcfcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:35.082048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.082012    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:35.083339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.083312    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy" pod "21bfd530-3cf3-4784-81f1-d78f9fb72192" (UID: "21bfd530-3cf3-4784-81f1-d78f9fb72192"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.179739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.179667    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcfcp\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-kube-api-access-qcfcp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.179739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.179692    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.179739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.179702    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.179739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.179714    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bfd530-3cf3-4784-81f1-d78f9fb72192-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.187441 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.187410    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" path="/var/lib/kubelet/pods/448f6a3f-07c9-4381-a356-ec466a740f46/volumes"
Oct 31 01:14:35.536235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.536201    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:14:35.583725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.583698    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:14:35.598112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598044    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598139    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598175    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598267    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598323    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598361    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598404    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598458    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598494    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598536    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598570    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598667    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598763    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598793    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598823    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.598984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598898    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token\") pod \"e08a8410-136a-4178-80ce-dd599ca42048\" (UID: \"e08a8410-136a-4178-80ce-dd599ca42048\") "
Oct 31 01:14:35.598984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.598940    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.599316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.599278    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.599494 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.599424    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket\") pod \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\" (UID: \"fda6944e-5abf-4cbb-a38d-1ebc269b6a05\") "
Oct 31 01:14:35.600690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.599854    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.600690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.600059    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.600690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.600231    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.600690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.599859    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.600690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.600303    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.600690 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.600367    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:35.601126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601051    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.601187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601120    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:35.601530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601394    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.601530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601451    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.601530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601480    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.601748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601729    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.601822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601758    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.601822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601772    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.601822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601788    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.601822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.601801    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08a8410-136a-4178-80ce-dd599ca42048-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.602594 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.602377    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:35.605162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.605140    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.605331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.605312    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "kube-api-access-wx6zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:35.605800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.605707    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:35.605800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.605775    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy" pod "e08a8410-136a-4178-80ce-dd599ca42048" (UID: "e08a8410-136a-4178-80ce-dd599ca42048"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:35.606116 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.605845    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:35.608772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.608743    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:35.608845 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.608777    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4" pod "fda6944e-5abf-4cbb-a38d-1ebc269b6a05" (UID: "fda6944e-5abf-4cbb-a38d-1ebc269b6a05"). InnerVolumeSpecName "kube-api-access-rnrp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:35.695882 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.695809    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerName="payload-processing" containerID="cri-o://63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d" gracePeriod=25
Oct 31 01:14:35.703385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703354    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703386    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703397    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703406    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703415    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703424    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703432    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08a8410-136a-4178-80ce-dd599ca42048-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703440    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703450    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnrp4\" (UniqueName: \"kubernetes.io/projected/fda6944e-5abf-4cbb-a38d-1ebc269b6a05-kube-api-access-rnrp4\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703465    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08a8410-136a-4178-80ce-dd599ca42048-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.703547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.703475    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx6zv\" (UniqueName: \"kubernetes.io/projected/e08a8410-136a-4178-80ce-dd599ca42048-kube-api-access-wx6zv\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:35.727746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.727707    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerName="payload-pre-processing" containerID="cri-o://cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d" gracePeriod=25
Oct 31 01:14:35.920983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.920928    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerName="payload-pre-processing" containerID="cri-o://b728f0bd2e62b70118895f063017471259f54bc6838c1841c583dd8c78a7e90e" gracePeriod=25
Oct 31 01:14:35.957407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.957375    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"
Oct 31 01:14:35.957953 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.957483    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk" event={"ID":"21bfd530-3cf3-4784-81f1-d78f9fb72192","Type":"ContainerDied","Data":"2e38adcf870cec2af1527e2b6b08759c510aa97201f0be903628aab258426943"}
Oct 31 01:14:35.957953 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.957582    8029 scope.go:122] "RemoveContainer" containerID="2701c160c1b02c9a3b1ead6165e7f18a40200af45c2bc316b8b020da3ceec1b7"
Oct 31 01:14:35.960715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.959905    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"
Oct 31 01:14:35.960715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.959956    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv" event={"ID":"fda6944e-5abf-4cbb-a38d-1ebc269b6a05","Type":"ContainerDied","Data":"3fc27c137068460ae38a793e933299ad9efb1520c7422408179ba3d1cf81340f"}
Oct 31 01:14:35.963394 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.963376    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"
Oct 31 01:14:35.968995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.968759    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk" event={"ID":"e08a8410-136a-4178-80ce-dd599ca42048","Type":"ContainerDied","Data":"a73e832e5fdcdd67c8b138cbc214dac42e4627f145973cd725a4a0ffb7041444"}
Oct 31 01:14:35.973470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.973453    8029 scope.go:122] "RemoveContainer" containerID="c1a18ecba2b13d0e00478b5bea8412b8a86c2f98368752b722e50f3961b505bf"
Oct 31 01:14:35.990582 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.990443    8029 scope.go:122] "RemoveContainer" containerID="6d602f1615bfaf5502bd86c375f265d4c4f1eb5a817bd74bccf52042a1730cfd"
Oct 31 01:14:36.001688 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:35.998556    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"]
Oct 31 01:14:36.007298 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.007231    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-vhznk"]
Oct 31 01:14:36.015633 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.015566    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"]
Oct 31 01:14:36.024742 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.024711    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-tbfnk"]
Oct 31 01:14:36.035658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.035629    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"]
Oct 31 01:14:36.044715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.044659    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0a6cca-openshift-default-7cf4f9647c-p2pjv"]
Oct 31 01:14:36.165269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.165241    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:36.210704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.210671    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca\") pod \"3c601844-fa2c-4865-befd-03f0dc788725\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") "
Oct 31 01:14:36.210855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.210828    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca\") pod \"3c601844-fa2c-4865-befd-03f0dc788725\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") "
Oct 31 01:14:36.210964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.210949    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q\" (UniqueName: \"kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q\") pod \"3c601844-fa2c-4865-befd-03f0dc788725\" (UID: \"3c601844-fa2c-4865-befd-03f0dc788725\") "
Oct 31 01:14:36.211836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.211810    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca" pod "3c601844-fa2c-4865-befd-03f0dc788725" (UID: "3c601844-fa2c-4865-befd-03f0dc788725"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:36.216122 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.216094    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q" pod "3c601844-fa2c-4865-befd-03f0dc788725" (UID: "3c601844-fa2c-4865-befd-03f0dc788725"). InnerVolumeSpecName "kube-api-access-dtw7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:36.218287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.218262    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca" pod "3c601844-fa2c-4865-befd-03f0dc788725" (UID: "3c601844-fa2c-4865-befd-03f0dc788725"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:36.245004 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.244976    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:36.248049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.248027    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:36.340685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.322905    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.340685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.322943    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtw7q\" (UniqueName: \"kubernetes.io/projected/3c601844-fa2c-4865-befd-03f0dc788725-kube-api-access-dtw7q\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.340685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.322960    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c601844-fa2c-4865-befd-03f0dc788725-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.423859 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.423823    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v\" (UniqueName: \"kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v\") pod \"743abb0c-6a7d-428d-9fae-c153f886345b\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") "
Oct 31 01:14:36.424048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.423882    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca\") pod \"743abb0c-6a7d-428d-9fae-c153f886345b\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") "
Oct 31 01:14:36.424048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.423983    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5\" (UniqueName: \"kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5\") pod \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") "
Oct 31 01:14:36.424048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424021    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca\") pod \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") "
Oct 31 01:14:36.424196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424130    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca\") pod \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\" (UID: \"3feb32fa-9d2d-4d52-b778-40e57217bbfa\") "
Oct 31 01:14:36.424196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424183    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca\") pod \"743abb0c-6a7d-428d-9fae-c153f886345b\" (UID: \"743abb0c-6a7d-428d-9fae-c153f886345b\") "
Oct 31 01:14:36.424989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424417    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca" pod "3feb32fa-9d2d-4d52-b778-40e57217bbfa" (UID: "3feb32fa-9d2d-4d52-b778-40e57217bbfa"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:36.424989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424713    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca" pod "743abb0c-6a7d-428d-9fae-c153f886345b" (UID: "743abb0c-6a7d-428d-9fae-c153f886345b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:36.424989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424944    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.424989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.424961    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.426367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.426337    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5" pod "3feb32fa-9d2d-4d52-b778-40e57217bbfa" (UID: "3feb32fa-9d2d-4d52-b778-40e57217bbfa"). InnerVolumeSpecName "kube-api-access-xp8x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:36.426493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.426447    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca" pod "3feb32fa-9d2d-4d52-b778-40e57217bbfa" (UID: "3feb32fa-9d2d-4d52-b778-40e57217bbfa"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:36.426802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.426781    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca" pod "743abb0c-6a7d-428d-9fae-c153f886345b" (UID: "743abb0c-6a7d-428d-9fae-c153f886345b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:36.426802 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.426786    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v" pod "743abb0c-6a7d-428d-9fae-c153f886345b" (UID: "743abb0c-6a7d-428d-9fae-c153f886345b"). InnerVolumeSpecName "kube-api-access-xxc8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:36.526073 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.525992    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxc8v\" (UniqueName: \"kubernetes.io/projected/743abb0c-6a7d-428d-9fae-c153f886345b-kube-api-access-xxc8v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.526073 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.526021    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/743abb0c-6a7d-428d-9fae-c153f886345b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.526073 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.526031    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp8x5\" (UniqueName: \"kubernetes.io/projected/3feb32fa-9d2d-4d52-b778-40e57217bbfa-kube-api-access-xp8x5\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.526073 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.526040    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3feb32fa-9d2d-4d52-b778-40e57217bbfa-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:36.977238 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.977210    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"
Oct 31 01:14:36.977698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.977380    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz" event={"ID":"3c601844-fa2c-4865-befd-03f0dc788725","Type":"ContainerDied","Data":"99c379ca0bbe11ad261cdf6665ee2cee1d5b96c3fdb465693ccfcd2684e0ba21"}
Oct 31 01:14:36.977698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.977440    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99c379ca0bbe11ad261cdf6665ee2cee1d5b96c3fdb465693ccfcd2684e0ba21"
Oct 31 01:14:36.981542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.980926    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"
Oct 31 01:14:36.981542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.980978    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628" event={"ID":"743abb0c-6a7d-428d-9fae-c153f886345b","Type":"ContainerDied","Data":"c95b27a9230c447f41a3158cd7097d96087e3b28fff3c2796961ab2b51da5885"}
Oct 31 01:14:36.981542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.981030    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c95b27a9230c447f41a3158cd7097d96087e3b28fff3c2796961ab2b51da5885"
Oct 31 01:14:36.984840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.984815    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz" event={"ID":"3feb32fa-9d2d-4d52-b778-40e57217bbfa","Type":"ContainerDied","Data":"3c1ee131c645484214f6f2074d7e42853c3b047f39a003c17c3ac0e51e91d741"}
Oct 31 01:14:36.984954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.984852    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"
Oct 31 01:14:36.984954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:36.984859    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c1ee131c645484214f6f2074d7e42853c3b047f39a003c17c3ac0e51e91d741"
Oct 31 01:14:37.188020 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:37.187985    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" path="/var/lib/kubelet/pods/21bfd530-3cf3-4784-81f1-d78f9fb72192/volumes"
Oct 31 01:14:37.188647 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:37.188624    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e08a8410-136a-4178-80ce-dd599ca42048" path="/var/lib/kubelet/pods/e08a8410-136a-4178-80ce-dd599ca42048/volumes"
Oct 31 01:14:37.189361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:37.189340    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" path="/var/lib/kubelet/pods/fda6944e-5abf-4cbb-a38d-1ebc269b6a05/volumes"
Oct 31 01:14:37.863662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:37.857653    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"]
Oct 31 01:14:37.883521 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:37.883484    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-8499c8-ss465"]
Oct 31 01:14:39.186265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:39.186232    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54cb07c4-233e-46d0-80df-57815272239b" path="/var/lib/kubelet/pods/54cb07c4-233e-46d0-80df-57815272239b/volumes"
Oct 31 01:14:40.181707 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.181676    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" containerID="cri-o://59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26" gracePeriod=45
Oct 31 01:14:40.203647 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.203623    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": dial tcp 10.134.0.87:8000: connect: connection refused"
Oct 31 01:14:40.220223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.220185    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" containerID="cri-o://5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb" gracePeriod=45
Oct 31 01:14:40.359626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.353880    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" containerID="cri-o://917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e" gracePeriod=45
Oct 31 01:14:40.363804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.362725    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"]
Oct 31 01:14:40.381654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.378708    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"]
Oct 31 01:14:40.565415 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.563224    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": dial tcp 10.134.0.59:8000: connect: connection refused"
Oct 31 01:14:40.578665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.578148    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:40.578665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.578392    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:40.579382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.579355    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:40.579703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.579584    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:40.602065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.601733    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:14:40.621549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.621207    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:14:40.701860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.701763    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir\") pod \"461dfedc-0264-4ba3-8248-b647aa593d60\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") "
Oct 31 01:14:40.702193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.702163    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir" pod "461dfedc-0264-4ba3-8248-b647aa593d60" (UID: "461dfedc-0264-4ba3-8248-b647aa593d60"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.702278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.702254    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs\" (UniqueName: \"kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs\") pod \"461dfedc-0264-4ba3-8248-b647aa593d60\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") "
Oct 31 01:14:40.702362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.702342    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m\" (UniqueName: \"kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m\") pod \"23154512-832c-4b95-bf6e-872707c8ec20\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") "
Oct 31 01:14:40.702803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.702780    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache\") pod \"23154512-832c-4b95-bf6e-872707c8ec20\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") "
Oct 31 01:14:40.702922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.702825    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj\" (UniqueName: \"kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj\") pod \"461dfedc-0264-4ba3-8248-b647aa593d60\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") "
Oct 31 01:14:40.702922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.702899    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm\") pod \"23154512-832c-4b95-bf6e-872707c8ec20\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") "
Oct 31 01:14:40.703049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703000    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home\") pod \"461dfedc-0264-4ba3-8248-b647aa593d60\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") "
Oct 31 01:14:40.703049 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703039    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache\") pod \"461dfedc-0264-4ba3-8248-b647aa593d60\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") "
Oct 31 01:14:40.703153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703093    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home\") pod \"23154512-832c-4b95-bf6e-872707c8ec20\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") "
Oct 31 01:14:40.703153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703145    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm\") pod \"461dfedc-0264-4ba3-8248-b647aa593d60\" (UID: \"461dfedc-0264-4ba3-8248-b647aa593d60\") "
Oct 31 01:14:40.703318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703180    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir\") pod \"23154512-832c-4b95-bf6e-872707c8ec20\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") "
Oct 31 01:14:40.703318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703199    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs\" (UniqueName: \"kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs\") pod \"23154512-832c-4b95-bf6e-872707c8ec20\" (UID: \"23154512-832c-4b95-bf6e-872707c8ec20\") "
Oct 31 01:14:40.703942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.703911    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home" pod "23154512-832c-4b95-bf6e-872707c8ec20" (UID: "23154512-832c-4b95-bf6e-872707c8ec20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.704051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.704029    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.704152 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.704128    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache" pod "461dfedc-0264-4ba3-8248-b647aa593d60" (UID: "461dfedc-0264-4ba3-8248-b647aa593d60"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.704251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.704232    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir" pod "23154512-832c-4b95-bf6e-872707c8ec20" (UID: "23154512-832c-4b95-bf6e-872707c8ec20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.704440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.704420    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache" pod "23154512-832c-4b95-bf6e-872707c8ec20" (UID: "23154512-832c-4b95-bf6e-872707c8ec20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.705014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.704993    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home" pod "461dfedc-0264-4ba3-8248-b647aa593d60" (UID: "461dfedc-0264-4ba3-8248-b647aa593d60"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.705240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.705207    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs" pod "461dfedc-0264-4ba3-8248-b647aa593d60" (UID: "461dfedc-0264-4ba3-8248-b647aa593d60"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:40.706309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.706285    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m" pod "23154512-832c-4b95-bf6e-872707c8ec20" (UID: "23154512-832c-4b95-bf6e-872707c8ec20"). InnerVolumeSpecName "kube-api-access-jzr8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:40.706844 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.706822    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj" pod "461dfedc-0264-4ba3-8248-b647aa593d60" (UID: "461dfedc-0264-4ba3-8248-b647aa593d60"). InnerVolumeSpecName "kube-api-access-xrfnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:40.708569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.708542    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm" pod "461dfedc-0264-4ba3-8248-b647aa593d60" (UID: "461dfedc-0264-4ba3-8248-b647aa593d60"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.708569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.708553    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm" pod "23154512-832c-4b95-bf6e-872707c8ec20" (UID: "23154512-832c-4b95-bf6e-872707c8ec20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.708722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.708589    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs" pod "23154512-832c-4b95-bf6e-872707c8ec20" (UID: "23154512-832c-4b95-bf6e-872707c8ec20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:40.717965 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.717945    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:14:40.805008 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.804976    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache\") pod \"2c4c3bf9-e242-4933-b54e-c38c035fced3\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") "
Oct 31 01:14:40.805171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805068    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs\") pod \"2c4c3bf9-e242-4933-b54e-c38c035fced3\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") "
Oct 31 01:14:40.805249 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805231    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w\" (UniqueName: \"kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w\") pod \"2c4c3bf9-e242-4933-b54e-c38c035fced3\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") "
Oct 31 01:14:40.805351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805335    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir\") pod \"2c4c3bf9-e242-4933-b54e-c38c035fced3\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") "
Oct 31 01:14:40.805434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805358    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache" pod "2c4c3bf9-e242-4933-b54e-c38c035fced3" (UID: "2c4c3bf9-e242-4933-b54e-c38c035fced3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.805498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805459    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home\") pod \"2c4c3bf9-e242-4933-b54e-c38c035fced3\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") "
Oct 31 01:14:40.805553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805515    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm\") pod \"2c4c3bf9-e242-4933-b54e-c38c035fced3\" (UID: \"2c4c3bf9-e242-4933-b54e-c38c035fced3\") "
Oct 31 01:14:40.806126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.805996    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir" pod "2c4c3bf9-e242-4933-b54e-c38c035fced3" (UID: "2c4c3bf9-e242-4933-b54e-c38c035fced3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.806239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806158    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home" pod "2c4c3bf9-e242-4933-b54e-c38c035fced3" (UID: "2c4c3bf9-e242-4933-b54e-c38c035fced3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806593    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806665    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806680    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806693    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806706    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/461dfedc-0264-4ba3-8248-b647aa593d60-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806720    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806733    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23154512-832c-4b95-bf6e-872707c8ec20-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806745    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806758    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/461dfedc-0264-4ba3-8248-b647aa593d60-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806773    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzr8m\" (UniqueName: \"kubernetes.io/projected/23154512-832c-4b95-bf6e-872707c8ec20-kube-api-access-jzr8m\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806787    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806801    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrfnj\" (UniqueName: \"kubernetes.io/projected/461dfedc-0264-4ba3-8248-b647aa593d60-kube-api-access-xrfnj\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806815    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23154512-832c-4b95-bf6e-872707c8ec20-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.806854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.806828    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.807969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.807948    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs" pod "2c4c3bf9-e242-4933-b54e-c38c035fced3" (UID: "2c4c3bf9-e242-4933-b54e-c38c035fced3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:40.808799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.808776    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm" pod "2c4c3bf9-e242-4933-b54e-c38c035fced3" (UID: "2c4c3bf9-e242-4933-b54e-c38c035fced3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:40.808949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.808807    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w" pod "2c4c3bf9-e242-4933-b54e-c38c035fced3" (UID: "2c4c3bf9-e242-4933-b54e-c38c035fced3"). InnerVolumeSpecName "kube-api-access-5277w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:40.908292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.908258    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5277w\" (UniqueName: \"kubernetes.io/projected/2c4c3bf9-e242-4933-b54e-c38c035fced3-kube-api-access-5277w\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.908292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.908287    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c4c3bf9-e242-4933-b54e-c38c035fced3-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:40.908292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:40.908298    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c4c3bf9-e242-4933-b54e-c38c035fced3-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:41.063214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.062275    8029 generic.go:356] "Generic (PLEG): container finished" podID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerID="917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e" exitCode=0
Oct 31 01:14:41.063214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.062392    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"
Oct 31 01:14:41.063214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.062370    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" event={"ID":"2c4c3bf9-e242-4933-b54e-c38c035fced3","Type":"ContainerDied","Data":"917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e"}
Oct 31 01:14:41.063214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.062477    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr" event={"ID":"2c4c3bf9-e242-4933-b54e-c38c035fced3","Type":"ContainerDied","Data":"13be766541065162ed55d85fdb61eb835e503af3b8b4616dcb89283ba95c5a16"}
Oct 31 01:14:41.063214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.062530    8029 scope.go:122] "RemoveContainer" containerID="917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e"
Oct 31 01:14:41.066184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.065850    8029 generic.go:356] "Generic (PLEG): container finished" podID="23154512-832c-4b95-bf6e-872707c8ec20" containerID="5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb" exitCode=0
Oct 31 01:14:41.066184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.065915    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"
Oct 31 01:14:41.066184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.066122    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" event={"ID":"23154512-832c-4b95-bf6e-872707c8ec20","Type":"ContainerDied","Data":"5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb"}
Oct 31 01:14:41.066184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.066158    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" event={"ID":"23154512-832c-4b95-bf6e-872707c8ec20","Type":"ContainerDied","Data":"b195a9abbd89a3e429c8fa684f6fe7b085ad8c0a9258fbc7f573f46e1d4cb41b"}
Oct 31 01:14:41.068335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.068311    8029 generic.go:356] "Generic (PLEG): container finished" podID="461dfedc-0264-4ba3-8248-b647aa593d60" containerID="59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26" exitCode=0
Oct 31 01:14:41.068468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.068357    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"
Oct 31 01:14:41.068468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.068415    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" event={"ID":"461dfedc-0264-4ba3-8248-b647aa593d60","Type":"ContainerDied","Data":"59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26"}
Oct 31 01:14:41.068468 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.068440    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw" event={"ID":"461dfedc-0264-4ba3-8248-b647aa593d60","Type":"ContainerDied","Data":"292a0c43a4e9f39bb278d88258b2b66edd7fd38bc124df262b41826003eb577c"}
Oct 31 01:14:41.097150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.096761    8029 scope.go:122] "RemoveContainer" containerID="917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e"
Oct 31 01:14:41.098296 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:41.098263    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e\": container with ID starting with 917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e not found: ID does not exist" containerID="917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e"
Oct 31 01:14:41.098405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.098314    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e"} err="failed to get container status \"917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e\": rpc error: code = NotFound desc = could not find container \"917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e\": container with ID starting with 917131fe592f2c7da8fba1353b6a09a229e4c816321fe021fabb4dd0f5ae6b6e not found: ID does not exist"
Oct 31 01:14:41.098405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.098339    8029 scope.go:122] "RemoveContainer" containerID="5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb"
Oct 31 01:14:41.113740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.113719    8029 scope.go:122] "RemoveContainer" containerID="5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb"
Oct 31 01:14:41.114033 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:41.114012    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb\": container with ID starting with 5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb not found: ID does not exist" containerID="5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb"
Oct 31 01:14:41.114115 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.114041    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb"} err="failed to get container status \"5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb\": rpc error: code = NotFound desc = could not find container \"5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb\": container with ID starting with 5cb2150ed0e9b3276685a1511f33b806e323a4c998a4413ee201479aab2651cb not found: ID does not exist"
Oct 31 01:14:41.114115 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.114061    8029 scope.go:122] "RemoveContainer" containerID="59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26"
Oct 31 01:14:41.116678 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.116655    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"]
Oct 31 01:14:41.121778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.121752    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-28ff08/premium-w5-28ff08-kserve-648ffb4f99-grpkr"]
Oct 31 01:14:41.126031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.126012    8029 scope.go:122] "RemoveContainer" containerID="59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26"
Oct 31 01:14:41.126299 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:41.126281    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26\": container with ID starting with 59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26 not found: ID does not exist" containerID="59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26"
Oct 31 01:14:41.126360 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.126305    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26"} err="failed to get container status \"59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26\": rpc error: code = NotFound desc = could not find container \"59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26\": container with ID starting with 59fd22dd197d1ade1af6dede7a6b9dc985f3e9d0a7c4d08c1cdc323cfa64bc26 not found: ID does not exist"
Oct 31 01:14:41.128549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.128532    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"]
Oct 31 01:14:41.132879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.132860    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/distinct-w1-7f6d1e-kserve-84988c4445-kp2fw"]
Oct 31 01:14:41.141116 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.141094    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"]
Oct 31 01:14:41.145145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.145128    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf"]
Oct 31 01:14:41.183516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.183488    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23154512-832c-4b95-bf6e-872707c8ec20" path="/var/lib/kubelet/pods/23154512-832c-4b95-bf6e-872707c8ec20/volumes"
Oct 31 01:14:41.183985 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.183969    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" path="/var/lib/kubelet/pods/2c4c3bf9-e242-4933-b54e-c38c035fced3/volumes"
Oct 31 01:14:41.184375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.184362    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" path="/var/lib/kubelet/pods/461dfedc-0264-4ba3-8248-b647aa593d60/volumes"
Oct 31 01:14:41.509295 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:41.509256    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7f6d1e/simulator-w1-7f6d1e-kserve-845d8b7ccf-b5bzf" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:43.037193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.037134    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:14:43.095508 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.095477    8029 generic.go:356] "Generic (PLEG): container finished" podID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerID="94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894" exitCode=0
Oct 31 01:14:43.095702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.095537    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"
Oct 31 01:14:43.095702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.095560    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" event={"ID":"32b93a2d-9fce-43a7-8ba0-d9e10053db09","Type":"ContainerDied","Data":"94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894"}
Oct 31 01:14:43.095702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.095650    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726" event={"ID":"32b93a2d-9fce-43a7-8ba0-d9e10053db09","Type":"ContainerDied","Data":"e76fc7ccb2c78051744dfba91e8751251c43ea6f7e1b6c5ab0e4df1a68932209"}
Oct 31 01:14:43.095702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.095684    8029 scope.go:122] "RemoveContainer" containerID="94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894"
Oct 31 01:14:43.111294 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.111270    8029 scope.go:122] "RemoveContainer" containerID="94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894"
Oct 31 01:14:43.111952 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:43.111703    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894\": container with ID starting with 94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894 not found: ID does not exist" containerID="94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894"
Oct 31 01:14:43.111952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.111742    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894"} err="failed to get container status \"94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894\": rpc error: code = NotFound desc = could not find container \"94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894\": container with ID starting with 94368b882d014bdb67ebf75f068b0ad53a7227a7e57c6a8e1aa45f6459bea894 not found: ID does not exist"
Oct 31 01:14:43.148855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.148832    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume\") pod \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") "
Oct 31 01:14:43.148980 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.148881    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx\" (UniqueName: \"kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx\") pod \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\" (UID: \"32b93a2d-9fce-43a7-8ba0-d9e10053db09\") "
Oct 31 01:14:43.149222 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.149200    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume" pod "32b93a2d-9fce-43a7-8ba0-d9e10053db09" (UID: "32b93a2d-9fce-43a7-8ba0-d9e10053db09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:43.149483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.149459    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b93a2d-9fce-43a7-8ba0-d9e10053db09-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:43.151220 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.151192    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx" pod "32b93a2d-9fce-43a7-8ba0-d9e10053db09" (UID: "32b93a2d-9fce-43a7-8ba0-d9e10053db09"). InnerVolumeSpecName "kube-api-access-jbgwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:43.250934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.250908    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbgwx\" (UniqueName: \"kubernetes.io/projected/32b93a2d-9fce-43a7-8ba0-d9e10053db09-kube-api-access-jbgwx\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:43.420031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.419986    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"]
Oct 31 01:14:43.425070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:43.425043    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-8499c8-d9b778c7c-6z726"]
Oct 31 01:14:45.248626 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.247848    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" path="/var/lib/kubelet/pods/32b93a2d-9fce-43a7-8ba0-d9e10053db09/volumes"
Oct 31 01:14:45.560694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.559882    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"]
Oct 31 01:14:45.560694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.560199    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" containerID="cri-o://ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa" gracePeriod=30
Oct 31 01:14:45.704628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.663670    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"]
Oct 31 01:14:45.704628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.663942    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" containerID="cri-o://9d11bb4f50dc3ebda1173575dbd694fdcd30d410288a6e0c2daa29634b1a624a" gracePeriod=30
Oct 31 01:14:45.847367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.847276    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"]
Oct 31 01:14:45.847833 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.847800    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" containerID="cri-o://0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b" gracePeriod=30
Oct 31 01:14:45.973492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.957881    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"]
Oct 31 01:14:45.973492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:45.958113    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" containerID="cri-o://6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5" gracePeriod=30
Oct 31 01:14:46.317641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.315965    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"]
Oct 31 01:14:46.561676 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.556786    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:14:46.561676 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.557360    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:14:46.693831 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.690150    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"]
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.697696    8029 cpu_manager.go:414] "containerMap: removing container" podUID="3feb32fa-9d2d-4d52-b778-40e57217bbfa" containerName="revoke-keys"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.697724    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3feb32fa-9d2d-4d52-b778-40e57217bbfa" containerName="revoke-keys"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.697762    8029 cpu_manager.go:414] "containerMap: removing container" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerName="payload-processing"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.697782    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerName="payload-processing"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.697947    8029 cpu_manager.go:414] "containerMap: removing container" podUID="743abb0c-6a7d-428d-9fae-c153f886345b" containerName="revoke-keys"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.697961    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="743abb0c-6a7d-428d-9fae-c153f886345b" containerName="revoke-keys"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698001    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy"
Oct 31 01:14:46.698012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698015    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698027    8029 cpu_manager.go:414] "containerMap: removing container" podUID="3c601844-fa2c-4865-befd-03f0dc788725" containerName="revoke-keys"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698034    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c601844-fa2c-4865-befd-03f0dc788725" containerName="revoke-keys"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698060    8029 cpu_manager.go:414] "containerMap: removing container" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" containerName="payload-processing"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698068    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" containerName="payload-processing"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698102    8029 cpu_manager.go:414] "containerMap: removing container" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698109    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698135    8029 cpu_manager.go:414] "containerMap: removing container" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698142    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698191    8029 cpu_manager.go:414] "containerMap: removing container" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698199    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698215    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698224    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698250    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main"
Oct 31 01:14:46.698516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698258    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698619    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="21bfd530-3cf3-4784-81f1-d78f9fb72192" containerName="istio-proxy"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698637    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698646    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="23154512-832c-4b95-bf6e-872707c8ec20" containerName="main"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698670    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="461dfedc-0264-4ba3-8248-b647aa593d60" containerName="main"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698688    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fda6944e-5abf-4cbb-a38d-1ebc269b6a05" containerName="istio-proxy"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698699    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c4c3bf9-e242-4933-b54e-c38c035fced3" containerName="main"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698718    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3feb32fa-9d2d-4d52-b778-40e57217bbfa" containerName="revoke-keys"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698734    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="32b93a2d-9fce-43a7-8ba0-d9e10053db09" containerName="payload-processing"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698755    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="743abb0c-6a7d-428d-9fae-c153f886345b" containerName="revoke-keys"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698764    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c601844-fa2c-4865-befd-03f0dc788725" containerName="revoke-keys"
Oct 31 01:14:46.699299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.698781    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="448f6a3f-07c9-4381-a356-ec466a740f46" containerName="payload-processing"
Oct 31 01:14:46.715779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.715751    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"]
Oct 31 01:14:46.716128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.716104    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.752800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.752063    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.752800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.752212    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.752800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.752292    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd6r2\" (UniqueName: \"kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.762239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.757301    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"]
Oct 31 01:14:46.762239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.758291    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy"
Oct 31 01:14:46.762239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.758310    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy"
Oct 31 01:14:46.762239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.758500    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e08a8410-136a-4178-80ce-dd599ca42048" containerName="istio-proxy"
Oct 31 01:14:46.817900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.799928    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:46.817900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.817401    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:14:46.817900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.817469    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:14:46.817900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.817541    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:14:46.817900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.817656    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:14:46.899337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.858162    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.899337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.858272    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd6r2\" (UniqueName: \"kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.899337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.858417    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.899337 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.898875    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"]
Oct 31 01:14:46.901129 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.901052    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.979694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.965371    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:46.979694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.965463    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:46.979694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.965750    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2wbc\" (UniqueName: \"kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:46.979694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.966740    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd6r2\" (UniqueName: \"kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:46.979694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:46.971751    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d46506d-fcf7v\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:47.072189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.072148    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2wbc\" (UniqueName: \"kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.072531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.072512    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.072734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.072715    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.075404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.075374    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.077040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.076996    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.084743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.084722    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2wbc\" (UniqueName: \"kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc\") pod \"maas-api-revoke-sub-04cde8ab-76h2c\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.215384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.211011    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:14:47.215384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.211125    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:47.228640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.219972    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:47.242402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.239901    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:14:47.242402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.240011    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:47.321742 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.321673    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:47.608233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.608190    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"]
Oct 31 01:14:47.617215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.617181    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-7f6d1e-p5srs"]
Oct 31 01:14:47.650544 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.650507    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"]
Oct 31 01:14:47.700040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:47.699943    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"]
Oct 31 01:14:47.724901 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:47.724854    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec2b33b_3e06_41a9_9c6d_a1a4e1604368.slice/crio-0bfce1fe2d67b2ca85ff7591e9f8457f2eda1210e6ba84511c4500907b3d40f0 WatchSource:0}: Error finding container 0bfce1fe2d67b2ca85ff7591e9f8457f2eda1210e6ba84511c4500907b3d40f0: Status 404 returned error can't find the container with id 0bfce1fe2d67b2ca85ff7591e9f8457f2eda1210e6ba84511c4500907b3d40f0
Oct 31 01:14:48.203485 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:48.203444    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v" event={"ID":"88a225ed-f649-47bf-8496-2ad4544dfd43","Type":"ContainerStarted","Data":"d5b6db4269ed28c706ad5db28e4176b05bcd2ad24146ecc606af787eff724884"}
Oct 31 01:14:48.205132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:48.205106    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c" event={"ID":"cec2b33b-3e06-41a9-9c6d-a1a4e1604368","Type":"ContainerStarted","Data":"0bfce1fe2d67b2ca85ff7591e9f8457f2eda1210e6ba84511c4500907b3d40f0"}
Oct 31 01:14:49.152715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.152681    8029 patch_prober.go:29] interesting pod/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:14:49.153099 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.152737    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:49.185531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.185505    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="417b23a9-09ea-4427-8f72-85aad51e5fc6" path="/var/lib/kubelet/pods/417b23a9-09ea-4427-8f72-85aad51e5fc6/volumes"
Oct 31 01:14:49.217019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.216983    8029 generic.go:356] "Generic (PLEG): container finished" podID="cec2b33b-3e06-41a9-9c6d-a1a4e1604368" containerID="db727b97c1f61c58b271d500a27492c506ee00233b39cca09008050cd8ca4c07" exitCode=0
Oct 31 01:14:49.217019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.217012    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c" event={"ID":"cec2b33b-3e06-41a9-9c6d-a1a4e1604368","Type":"ContainerDied","Data":"db727b97c1f61c58b271d500a27492c506ee00233b39cca09008050cd8ca4c07"}
Oct 31 01:14:49.219062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.219034    8029 generic.go:356] "Generic (PLEG): container finished" podID="88a225ed-f649-47bf-8496-2ad4544dfd43" containerID="55cc2ef78c8a0f2b6f0d1087c1cf68730aa693b8422a4b24e992267dfbd764e1" exitCode=0
Oct 31 01:14:49.219214 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:49.219069    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v" event={"ID":"88a225ed-f649-47bf-8496-2ad4544dfd43","Type":"ContainerDied","Data":"55cc2ef78c8a0f2b6f0d1087c1cf68730aa693b8422a4b24e992267dfbd764e1"}
Oct 31 01:14:49.557349 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:49.557264    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:14:50.475646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.475583    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:50.478410 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.478389    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:50.554782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.554695    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc\" (UniqueName: \"kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc\") pod \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") "
Oct 31 01:14:50.554782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.554763    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca\") pod \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") "
Oct 31 01:14:50.555029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.554878    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2\" (UniqueName: \"kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2\") pod \"88a225ed-f649-47bf-8496-2ad4544dfd43\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") "
Oct 31 01:14:50.555029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.554917    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca\") pod \"88a225ed-f649-47bf-8496-2ad4544dfd43\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") "
Oct 31 01:14:50.555029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.555013    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca\") pod \"88a225ed-f649-47bf-8496-2ad4544dfd43\" (UID: \"88a225ed-f649-47bf-8496-2ad4544dfd43\") "
Oct 31 01:14:50.555209 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.555044    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca\") pod \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\" (UID: \"cec2b33b-3e06-41a9-9c6d-a1a4e1604368\") "
Oct 31 01:14:50.555209 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.555104    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca" pod "cec2b33b-3e06-41a9-9c6d-a1a4e1604368" (UID: "cec2b33b-3e06-41a9-9c6d-a1a4e1604368"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:50.555328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.555304    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca" pod "88a225ed-f649-47bf-8496-2ad4544dfd43" (UID: "88a225ed-f649-47bf-8496-2ad4544dfd43"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:50.556329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.556165    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:50.556329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.556195    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:50.557434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.557410    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2" pod "88a225ed-f649-47bf-8496-2ad4544dfd43" (UID: "88a225ed-f649-47bf-8496-2ad4544dfd43"). InnerVolumeSpecName "kube-api-access-jd6r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:50.557586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.557558    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca" pod "cec2b33b-3e06-41a9-9c6d-a1a4e1604368" (UID: "cec2b33b-3e06-41a9-9c6d-a1a4e1604368"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:50.557768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.557746    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc" pod "cec2b33b-3e06-41a9-9c6d-a1a4e1604368" (UID: "cec2b33b-3e06-41a9-9c6d-a1a4e1604368"). InnerVolumeSpecName "kube-api-access-d2wbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:50.557947 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.557918    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca" pod "88a225ed-f649-47bf-8496-2ad4544dfd43" (UID: "88a225ed-f649-47bf-8496-2ad4544dfd43"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:50.566910 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.566877    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:50.567036 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.566999    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:50.567153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.567132    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:14:50.657651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.657615    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd6r2\" (UniqueName: \"kubernetes.io/projected/88a225ed-f649-47bf-8496-2ad4544dfd43-kube-api-access-jd6r2\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:50.657651 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.657651    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/88a225ed-f649-47bf-8496-2ad4544dfd43-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:50.657850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.657661    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:50.657850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:50.657671    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2wbc\" (UniqueName: \"kubernetes.io/projected/cec2b33b-3e06-41a9-9c6d-a1a4e1604368-kube-api-access-d2wbc\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.268847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.268286    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v" event={"ID":"88a225ed-f649-47bf-8496-2ad4544dfd43","Type":"ContainerDied","Data":"d5b6db4269ed28c706ad5db28e4176b05bcd2ad24146ecc606af787eff724884"}
Oct 31 01:14:51.268847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.268352    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5b6db4269ed28c706ad5db28e4176b05bcd2ad24146ecc606af787eff724884"
Oct 31 01:14:51.268847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.268304    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"
Oct 31 01:14:51.271528 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.271504    8029 generic.go:356] "Generic (PLEG): container finished" podID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerID="9d11bb4f50dc3ebda1173575dbd694fdcd30d410288a6e0c2daa29634b1a624a" exitCode=0
Oct 31 01:14:51.271702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.271593    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" event={"ID":"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92","Type":"ContainerDied","Data":"9d11bb4f50dc3ebda1173575dbd694fdcd30d410288a6e0c2daa29634b1a624a"}
Oct 31 01:14:51.271702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.271686    8029 scope.go:122] "RemoveContainer" containerID="ca93755bd79f7014cd5cf41390388570a779ef885e311f92b88d8e5a6127540e"
Oct 31 01:14:51.273376 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.273357    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"
Oct 31 01:14:51.273465 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.273373    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c" event={"ID":"cec2b33b-3e06-41a9-9c6d-a1a4e1604368","Type":"ContainerDied","Data":"0bfce1fe2d67b2ca85ff7591e9f8457f2eda1210e6ba84511c4500907b3d40f0"}
Oct 31 01:14:51.273465 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.273434    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bfce1fe2d67b2ca85ff7591e9f8457f2eda1210e6ba84511c4500907b3d40f0"
Oct 31 01:14:51.303630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.303551    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:14:51.316714 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.316335    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerName="payload-processing" containerID="cri-o://6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9" gracePeriod=25
Oct 31 01:14:51.471368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471338    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471379    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471419    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471538    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471592    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471691 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471663    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471960 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471697    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471960 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471759    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.471960 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.471834    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket\") pod \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\" (UID: \"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92\") "
Oct 31 01:14:51.472472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.472277    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.472544 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.472463    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.473450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.472853    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.473450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.472879    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.473450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.472996    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.473450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.473092    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.473450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.473409    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:51.483781 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.483747    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:51.483781 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.483755    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:51.484257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.483795    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.484257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.483851    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls" pod "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" (UID: "fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92"). InnerVolumeSpecName "kube-api-access-fnbls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:51.504265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.504241    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573784    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573813    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573822    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573835    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573849    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573860    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.573866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.573869    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnbls\" (UniqueName: \"kubernetes.io/projected/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92-kube-api-access-fnbls\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.677107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677046    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677308 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677167    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677308 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677247    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677370    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677421    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677484    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677548    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677635    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677678    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data\") pod \"d2813b0a-d570-425a-9523-778af9495993\" (UID: \"d2813b0a-d570-425a-9523-778af9495993\") "
Oct 31 01:14:51.677789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.677764    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.678197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.678170    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.678520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.678215    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.682147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682120    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.682274 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682246    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "kube-api-access-xkj9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:51.682487 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682465    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:51.682719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682689    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkj9v\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-kube-api-access-xkj9v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.682814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682726    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.682814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682742    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.682814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682756    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2813b0a-d570-425a-9523-778af9495993-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.682814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682769    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.682814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.682784    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.683367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.683328    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:51.684068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.684023    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:51.684906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.684885    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token" pod "d2813b0a-d570-425a-9523-778af9495993" (UID: "d2813b0a-d570-425a-9523-778af9495993"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:51.787120 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.784801    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2813b0a-d570-425a-9523-778af9495993-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.787120 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.784838    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2813b0a-d570-425a-9523-778af9495993-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.787120 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.784853    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2813b0a-d570-425a-9523-778af9495993-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:51.921576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.921549    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:14:51.986835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:51.986794    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:14:52.088780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.088752    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.088942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.088818    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.088997    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089033    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089116 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089066    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089116 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089109    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089180 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089141    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089236 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089183    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089236 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089212    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089332 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089290    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089338    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089380    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089489 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089458    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089576    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089644    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089651    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.089744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089720    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.089974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089751    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket\") pod \"93013175-f94f-4939-8d9d-78930d529213\" (UID: \"93013175-f94f-4939-8d9d-78930d529213\") "
Oct 31 01:14:52.089974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089923    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.090091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.089989    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.090091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090000    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert\") pod \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\" (UID: \"d23822fb-a22c-4abe-8fb3-abe2879cdef9\") "
Oct 31 01:14:52.090199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090133    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090468    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090753    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090938    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090964    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090980    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.090995    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.091009    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.091293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.091023    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93013175-f94f-4939-8d9d-78930d529213-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.091887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.091697    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:52.091887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.091881    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.092024 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.091927    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.092208 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.092151    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.092410 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.092230    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:52.092410 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.092341    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.092410 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.092353    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "kube-api-access-wxmcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:52.092592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.092451    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:52.093130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.093104    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:52.093463 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.093445    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "kube-api-access-pzgxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:52.093706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.093688    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo" pod "d23822fb-a22c-4abe-8fb3-abe2879cdef9" (UID: "d23822fb-a22c-4abe-8fb3-abe2879cdef9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:52.094110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.094087    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo" pod "93013175-f94f-4939-8d9d-78930d529213" (UID: "93013175-f94f-4939-8d9d-78930d529213"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192310    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192341    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192352    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d23822fb-a22c-4abe-8fb3-abe2879cdef9-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192360    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93013175-f94f-4939-8d9d-78930d529213-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192370    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxmcm\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-kube-api-access-wxmcm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192382    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192390    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192399    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzgxp\" (UniqueName: \"kubernetes.io/projected/d23822fb-a22c-4abe-8fb3-abe2879cdef9-kube-api-access-pzgxp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192407    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192415    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d23822fb-a22c-4abe-8fb3-abe2879cdef9-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192425    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93013175-f94f-4939-8d9d-78930d529213-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.192746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.192433    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93013175-f94f-4939-8d9d-78930d529213-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:52.283210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.283172    8029 generic.go:356] "Generic (PLEG): container finished" podID="93013175-f94f-4939-8d9d-78930d529213" containerID="6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5" exitCode=0
Oct 31 01:14:52.283396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.283224    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"
Oct 31 01:14:52.283396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.283250    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" event={"ID":"93013175-f94f-4939-8d9d-78930d529213","Type":"ContainerDied","Data":"6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5"}
Oct 31 01:14:52.283396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.283301    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78" event={"ID":"93013175-f94f-4939-8d9d-78930d529213","Type":"ContainerDied","Data":"e5ddcd3bfb54c1fdc5c3bbb6b76ddf8e0f00d9c59f56b3a6aaf71f50c90e28d7"}
Oct 31 01:14:52.283396 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.283327    8029 scope.go:122] "RemoveContainer" containerID="6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5"
Oct 31 01:14:52.285069 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.285044    8029 generic.go:356] "Generic (PLEG): container finished" podID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerID="ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa" exitCode=0
Oct 31 01:14:52.285166 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.285128    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" event={"ID":"d23822fb-a22c-4abe-8fb3-abe2879cdef9","Type":"ContainerDied","Data":"ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa"}
Oct 31 01:14:52.285223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.285189    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk" event={"ID":"d23822fb-a22c-4abe-8fb3-abe2879cdef9","Type":"ContainerDied","Data":"74541639db952a09e033d5dd3f4b094b7ccbb47d2ab95760d12d4443db9a3658"}
Oct 31 01:14:52.285291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.285230    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"
Oct 31 01:14:52.292012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.291123    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"
Oct 31 01:14:52.292012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.291146    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8" event={"ID":"fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92","Type":"ContainerDied","Data":"c90605c480c5505a72cea84d8d37c2f9fef12e160311c3a2e058af52b5c67368"}
Oct 31 01:14:52.300283 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.300259    8029 scope.go:122] "RemoveContainer" containerID="6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5"
Oct 31 01:14:52.300939 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:52.300910    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5\": container with ID starting with 6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5 not found: ID does not exist" containerID="6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5"
Oct 31 01:14:52.301047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.300949    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5"} err="failed to get container status \"6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5\": rpc error: code = NotFound desc = could not find container \"6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5\": container with ID starting with 6fd0149d40a10a87c587c4601859190dc8180da7dd9dc62727aa7fbad87cc7b5 not found: ID does not exist"
Oct 31 01:14:52.301047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.300971    8029 scope.go:122] "RemoveContainer" containerID="ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa"
Oct 31 01:14:52.301192 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.301163    8029 generic.go:356] "Generic (PLEG): container finished" podID="d2813b0a-d570-425a-9523-778af9495993" containerID="0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b" exitCode=0
Oct 31 01:14:52.301245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.301210    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"
Oct 31 01:14:52.301245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.301233    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" event={"ID":"d2813b0a-d570-425a-9523-778af9495993","Type":"ContainerDied","Data":"0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b"}
Oct 31 01:14:52.301350 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.301272    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7" event={"ID":"d2813b0a-d570-425a-9523-778af9495993","Type":"ContainerDied","Data":"3f83f7da5dce113eb90cc0c8941f7b67f2d36397d4f397af72ae77f9b80615d0"}
Oct 31 01:14:52.314989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.314833    8029 scope.go:122] "RemoveContainer" containerID="ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa"
Oct 31 01:14:52.315699 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:52.315671    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa\": container with ID starting with ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa not found: ID does not exist" containerID="ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa"
Oct 31 01:14:52.315764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.315712    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa"} err="failed to get container status \"ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa\": rpc error: code = NotFound desc = could not find container \"ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa\": container with ID starting with ddf54a4b2b441843907a4c91e6ad7b27bf2969c4176e5089840b0b14d724cbfa not found: ID does not exist"
Oct 31 01:14:52.315764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.315735    8029 scope.go:122] "RemoveContainer" containerID="9d11bb4f50dc3ebda1173575dbd694fdcd30d410288a6e0c2daa29634b1a624a"
Oct 31 01:14:52.327964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.327940    8029 scope.go:122] "RemoveContainer" containerID="0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b"
Oct 31 01:14:52.340288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.340266    8029 scope.go:122] "RemoveContainer" containerID="0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b"
Oct 31 01:14:52.340647 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:52.340630    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b\": container with ID starting with 0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b not found: ID does not exist" containerID="0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b"
Oct 31 01:14:52.340718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.340654    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b"} err="failed to get container status \"0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b\": rpc error: code = NotFound desc = could not find container \"0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b\": container with ID starting with 0f5734a9ea48b11c27ded6c8285438999bdc20c1b7b852edaf916e360a23f43b not found: ID does not exist"
Oct 31 01:14:52.352368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.352347    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"]
Oct 31 01:14:52.357339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.357315    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-55qp7"]
Oct 31 01:14:52.364300 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.364215    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"]
Oct 31 01:14:52.369542 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.369519    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-cdb78"]
Oct 31 01:14:52.380899 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.380872    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"]
Oct 31 01:14:52.385964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.385937    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-s48b8"]
Oct 31 01:14:52.395725 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.395697    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"]
Oct 31 01:14:52.400142 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:52.400119    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-8499c8-openshift-default-8587968855-jhqkk"]
Oct 31 01:14:53.185352 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:53.185321    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93013175-f94f-4939-8d9d-78930d529213" path="/var/lib/kubelet/pods/93013175-f94f-4939-8d9d-78930d529213/volumes"
Oct 31 01:14:53.185851 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:53.185836    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" path="/var/lib/kubelet/pods/d23822fb-a22c-4abe-8fb3-abe2879cdef9/volumes"
Oct 31 01:14:53.186309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:53.186295    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2813b0a-d570-425a-9523-778af9495993" path="/var/lib/kubelet/pods/d2813b0a-d570-425a-9523-778af9495993/volumes"
Oct 31 01:14:53.187031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:53.187015    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" path="/var/lib/kubelet/pods/fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92/volumes"
Oct 31 01:14:55.081625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.077852    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"]
Oct 31 01:14:55.081625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.078134    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy" containerID="cri-o://9a887b9d2170ef5204f0c57358f1bc1ddab4e6d1562ef268cbd2c1269ba7295c" gracePeriod=30
Oct 31 01:14:55.091397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.090976    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"]
Oct 31 01:14:55.091397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.091215    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" containerID="cri-o://f7705e9036e1e00bb902d4234c576470d6bdc73483ea0127860fb772664c48ba" gracePeriod=30
Oct 31 01:14:55.111506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.111465    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"]
Oct 31 01:14:55.111770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.111699    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" containerID="cri-o://4ad5b72a515e25c3ef76e605fdad322bcb1ac4d52b1568ca137895dc26c0f5d9" gracePeriod=30
Oct 31 01:14:55.114489 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.114461    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"]
Oct 31 01:14:55.114838 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.114776    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy" containerID="cri-o://767f37acc4134f0758a740ea3ec64a4d2dc223f8c4984af5525b8045882fdf06" gracePeriod=30
Oct 31 01:14:55.397263 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.397215    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main" containerID="cri-o://506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585" gracePeriod=45
Oct 31 01:14:55.422420 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.422368    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main" containerID="cri-o://42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07" gracePeriod=45
Oct 31 01:14:55.865509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.865471    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:14:55.878624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.878470    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:14:55.985205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985177    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t\" (UniqueName: \"kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t\") pod \"ad74513f-78eb-4047-9421-79740761bbd6\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") "
Oct 31 01:14:55.985326 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985310    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache\") pod \"ad74513f-78eb-4047-9421-79740761bbd6\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") "
Oct 31 01:14:55.985378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985357    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home\") pod \"25a79474-3747-4ba2-b31d-754962b8f9e1\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") "
Oct 31 01:14:55.985418 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985396    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm\") pod \"ad74513f-78eb-4047-9421-79740761bbd6\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") "
Oct 31 01:14:55.985509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985417    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs\" (UniqueName: \"kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs\") pod \"25a79474-3747-4ba2-b31d-754962b8f9e1\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") "
Oct 31 01:14:55.985509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985449    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir\") pod \"ad74513f-78eb-4047-9421-79740761bbd6\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") "
Oct 31 01:14:55.985509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985469    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm\") pod \"25a79474-3747-4ba2-b31d-754962b8f9e1\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") "
Oct 31 01:14:55.985509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985500    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache\") pod \"25a79474-3747-4ba2-b31d-754962b8f9e1\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") "
Oct 31 01:14:55.985722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985556    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home\") pod \"ad74513f-78eb-4047-9421-79740761bbd6\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") "
Oct 31 01:14:55.985722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985656    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss\" (UniqueName: \"kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss\") pod \"25a79474-3747-4ba2-b31d-754962b8f9e1\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") "
Oct 31 01:14:55.985722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985689    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs\" (UniqueName: \"kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs\") pod \"ad74513f-78eb-4047-9421-79740761bbd6\" (UID: \"ad74513f-78eb-4047-9421-79740761bbd6\") "
Oct 31 01:14:55.985722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.985721    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir\") pod \"25a79474-3747-4ba2-b31d-754962b8f9e1\" (UID: \"25a79474-3747-4ba2-b31d-754962b8f9e1\") "
Oct 31 01:14:55.986941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.986912    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home" pod "ad74513f-78eb-4047-9421-79740761bbd6" (UID: "ad74513f-78eb-4047-9421-79740761bbd6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:55.987643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.987129    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache" pod "ad74513f-78eb-4047-9421-79740761bbd6" (UID: "ad74513f-78eb-4047-9421-79740761bbd6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:55.987643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.987227    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache" pod "25a79474-3747-4ba2-b31d-754962b8f9e1" (UID: "25a79474-3747-4ba2-b31d-754962b8f9e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:55.987643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.987280    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home" pod "25a79474-3747-4ba2-b31d-754962b8f9e1" (UID: "25a79474-3747-4ba2-b31d-754962b8f9e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:55.988143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.988117    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir" pod "ad74513f-78eb-4047-9421-79740761bbd6" (UID: "ad74513f-78eb-4047-9421-79740761bbd6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:55.988312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.988181    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir" pod "25a79474-3747-4ba2-b31d-754962b8f9e1" (UID: "25a79474-3747-4ba2-b31d-754962b8f9e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:55.997216 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:55.997164    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs" pod "ad74513f-78eb-4047-9421-79740761bbd6" (UID: "ad74513f-78eb-4047-9421-79740761bbd6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:56.003518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.003486    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs" pod "25a79474-3747-4ba2-b31d-754962b8f9e1" (UID: "25a79474-3747-4ba2-b31d-754962b8f9e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:14:56.003704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.003491    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm" pod "ad74513f-78eb-4047-9421-79740761bbd6" (UID: "ad74513f-78eb-4047-9421-79740761bbd6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:56.003704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.003650    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm" pod "25a79474-3747-4ba2-b31d-754962b8f9e1" (UID: "25a79474-3747-4ba2-b31d-754962b8f9e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:14:56.003828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.003781    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t" pod "ad74513f-78eb-4047-9421-79740761bbd6" (UID: "ad74513f-78eb-4047-9421-79740761bbd6"). InnerVolumeSpecName "kube-api-access-lm94t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:56.005863 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.005836    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss" pod "25a79474-3747-4ba2-b31d-754962b8f9e1" (UID: "25a79474-3747-4ba2-b31d-754962b8f9e1"). InnerVolumeSpecName "kube-api-access-8smss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:56.080173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.079594    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"]
Oct 31 01:14:56.080173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.079881    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" containerID="cri-o://ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411" gracePeriod=30
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088653    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088683    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25a79474-3747-4ba2-b31d-754962b8f9e1-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088697    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088711    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088724    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088738    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088752    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8smss\" (UniqueName: \"kubernetes.io/projected/25a79474-3747-4ba2-b31d-754962b8f9e1-kube-api-access-8smss\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088765    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad74513f-78eb-4047-9421-79740761bbd6-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088777    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088790    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm94t\" (UniqueName: \"kubernetes.io/projected/ad74513f-78eb-4047-9421-79740761bbd6-kube-api-access-lm94t\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088801    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad74513f-78eb-4047-9421-79740761bbd6-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.089276 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.088813    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25a79474-3747-4ba2-b31d-754962b8f9e1-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.093132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.091535    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"]
Oct 31 01:14:56.093132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.091922    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" containerID="cri-o://1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1" gracePeriod=30
Oct 31 01:14:56.105721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.105691    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"]
Oct 31 01:14:56.106286 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.105937    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy" containerID="cri-o://9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053" gracePeriod=30
Oct 31 01:14:56.116325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.110794    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:14:56.116767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.116734    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:14:56.190881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.190387    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5\" (UniqueName: \"kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5\") pod \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") "
Oct 31 01:14:56.190881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.190448    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m\" (UniqueName: \"kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m\") pod \"5b8b9992-fca7-49f1-b464-b445d8054d5d\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") "
Oct 31 01:14:56.190881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.190539    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume\") pod \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\" (UID: \"bafc302a-4d33-44df-b0b2-29fdb44a84b8\") "
Oct 31 01:14:56.195120 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.195085    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume" pod "bafc302a-4d33-44df-b0b2-29fdb44a84b8" (UID: "bafc302a-4d33-44df-b0b2-29fdb44a84b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:56.196872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.196847    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m" pod "5b8b9992-fca7-49f1-b464-b445d8054d5d" (UID: "5b8b9992-fca7-49f1-b464-b445d8054d5d"). InnerVolumeSpecName "kube-api-access-2nr9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:56.263799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.258130    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5" pod "bafc302a-4d33-44df-b0b2-29fdb44a84b8" (UID: "bafc302a-4d33-44df-b0b2-29fdb44a84b8"). InnerVolumeSpecName "kube-api-access-27mx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:56.296128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.294349    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume\") pod \"5b8b9992-fca7-49f1-b464-b445d8054d5d\" (UID: \"5b8b9992-fca7-49f1-b464-b445d8054d5d\") "
Oct 31 01:14:56.296128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.294976    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27mx5\" (UniqueName: \"kubernetes.io/projected/bafc302a-4d33-44df-b0b2-29fdb44a84b8-kube-api-access-27mx5\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.296128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.295001    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nr9m\" (UniqueName: \"kubernetes.io/projected/5b8b9992-fca7-49f1-b464-b445d8054d5d-kube-api-access-2nr9m\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.296128 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.295015    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafc302a-4d33-44df-b0b2-29fdb44a84b8-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.299143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.299110    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume" pod "5b8b9992-fca7-49f1-b464-b445d8054d5d" (UID: "5b8b9992-fca7-49f1-b464-b445d8054d5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:56.345895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.345854    8029 generic.go:356] "Generic (PLEG): container finished" podID="ad74513f-78eb-4047-9421-79740761bbd6" containerID="506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585" exitCode=0
Oct 31 01:14:56.346083 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.345906    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" event={"ID":"ad74513f-78eb-4047-9421-79740761bbd6","Type":"ContainerDied","Data":"506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585"}
Oct 31 01:14:56.346083 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.345977    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7" event={"ID":"ad74513f-78eb-4047-9421-79740761bbd6","Type":"ContainerDied","Data":"9f5233eda531782288296b3ab439856ca43608e7232685ea03cf939db171ec80"}
Oct 31 01:14:56.346083 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.345984    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"
Oct 31 01:14:56.346083 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.346027    8029 scope.go:122] "RemoveContainer" containerID="506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585"
Oct 31 01:14:56.348268 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.348227    8029 generic.go:356] "Generic (PLEG): container finished" podID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerID="42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07" exitCode=0
Oct 31 01:14:56.348432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.348287    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"
Oct 31 01:14:56.348432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.348378    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" event={"ID":"25a79474-3747-4ba2-b31d-754962b8f9e1","Type":"ContainerDied","Data":"42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07"}
Oct 31 01:14:56.348432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.348409    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9" event={"ID":"25a79474-3747-4ba2-b31d-754962b8f9e1","Type":"ContainerDied","Data":"9cc7ae6bd67c7ed6759f01987f5690ad2abf7af18befdf8d7b6479abb71616e2"}
Oct 31 01:14:56.350720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.350695    8029 generic.go:356] "Generic (PLEG): container finished" podID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerID="63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d" exitCode=0
Oct 31 01:14:56.351026 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.351007    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" event={"ID":"5b8b9992-fca7-49f1-b464-b445d8054d5d","Type":"ContainerDied","Data":"63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d"}
Oct 31 01:14:56.351171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.351148    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps" event={"ID":"5b8b9992-fca7-49f1-b464-b445d8054d5d","Type":"ContainerDied","Data":"796fdcac8b34aa55fefbbf8622a3e23329a29f1300ae9fc8f7be10fe888ad7f6"}
Oct 31 01:14:56.351270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.351251    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"
Oct 31 01:14:56.354847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.354824    8029 generic.go:356] "Generic (PLEG): container finished" podID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerID="cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d" exitCode=0
Oct 31 01:14:56.354954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.354905    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"
Oct 31 01:14:56.354954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.354945    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" event={"ID":"bafc302a-4d33-44df-b0b2-29fdb44a84b8","Type":"ContainerDied","Data":"cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d"}
Oct 31 01:14:56.355089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.354986    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb" event={"ID":"bafc302a-4d33-44df-b0b2-29fdb44a84b8","Type":"ContainerDied","Data":"32325d478f867c2464df82cc8ff3d7cbda8282ed1d330068563beee6b8899c69"}
Oct 31 01:14:56.359451 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.359405    8029 generic.go:356] "Generic (PLEG): container finished" podID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerID="b728f0bd2e62b70118895f063017471259f54bc6838c1841c583dd8c78a7e90e" exitCode=0
Oct 31 01:14:56.359525 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.359480    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" event={"ID":"957a9635-2910-4b1b-9cc9-87e7672971f5","Type":"ContainerDied","Data":"b728f0bd2e62b70118895f063017471259f54bc6838c1841c583dd8c78a7e90e"}
Oct 31 01:14:56.364642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.364580    8029 scope.go:122] "RemoveContainer" containerID="506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585"
Oct 31 01:14:56.364870 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:56.364851    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585\": container with ID starting with 506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585 not found: ID does not exist" containerID="506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585"
Oct 31 01:14:56.364933 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.364878    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585"} err="failed to get container status \"506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585\": rpc error: code = NotFound desc = could not find container \"506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585\": container with ID starting with 506be67e6569114aa4aa1d7e273724dbade0d9411106935e4b6633ff72dc1585 not found: ID does not exist"
Oct 31 01:14:56.364933 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.364895    8029 scope.go:122] "RemoveContainer" containerID="42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07"
Oct 31 01:14:56.365211 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.365192    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:14:56.378538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.378518    8029 scope.go:122] "RemoveContainer" containerID="42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07"
Oct 31 01:14:56.378799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.378781    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"]
Oct 31 01:14:56.378875 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:56.378837    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07\": container with ID starting with 42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07 not found: ID does not exist" containerID="42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07"
Oct 31 01:14:56.378932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.378874    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07"} err="failed to get container status \"42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07\": rpc error: code = NotFound desc = could not find container \"42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07\": container with ID starting with 42f637b93d32d1fa34463c38aaa57985805a53e5afa23d9a12e78df34df98f07 not found: ID does not exist"
Oct 31 01:14:56.378932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.378895    8029 scope.go:122] "RemoveContainer" containerID="63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d"
Oct 31 01:14:56.388930 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.388910    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/distinct-w0-acdaed-kserve-845c67f5c9-qh9k7"]
Oct 31 01:14:56.389348 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.389336    8029 scope.go:122] "RemoveContainer" containerID="63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d"
Oct 31 01:14:56.389654 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:56.389595    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d\": container with ID starting with 63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d not found: ID does not exist" containerID="63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d"
Oct 31 01:14:56.389705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.389661    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d"} err="failed to get container status \"63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d\": rpc error: code = NotFound desc = could not find container \"63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d\": container with ID starting with 63972fe37a0410819f56de0d93e83a36b2ef4618da1d461e9fc7c661d362a51d not found: ID does not exist"
Oct 31 01:14:56.389705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.389676    8029 scope.go:122] "RemoveContainer" containerID="cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d"
Oct 31 01:14:56.396041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.396021    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume\") pod \"957a9635-2910-4b1b-9cc9-87e7672971f5\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") "
Oct 31 01:14:56.396674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.396655    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng\" (UniqueName: \"kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng\") pod \"957a9635-2910-4b1b-9cc9-87e7672971f5\" (UID: \"957a9635-2910-4b1b-9cc9-87e7672971f5\") "
Oct 31 01:14:56.397330 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.397312    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8b9992-fca7-49f1-b464-b445d8054d5d-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.397741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.396463    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume" pod "957a9635-2910-4b1b-9cc9-87e7672971f5" (UID: "957a9635-2910-4b1b-9cc9-87e7672971f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:14:56.399974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.399953    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng" pod "957a9635-2910-4b1b-9cc9-87e7672971f5" (UID: "957a9635-2910-4b1b-9cc9-87e7672971f5"). InnerVolumeSpecName "kube-api-access-2wqng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:14:56.403780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.403759    8029 scope.go:122] "RemoveContainer" containerID="cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d"
Oct 31 01:14:56.410165 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:14:56.410132    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d\": container with ID starting with cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d not found: ID does not exist" containerID="cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d"
Oct 31 01:14:56.410256 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.410177    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d"} err="failed to get container status \"cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d\": rpc error: code = NotFound desc = could not find container \"cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d\": container with ID starting with cee16aa7d7be52291c4a9183bc88cc9bb6235147a206e1c7ad5319ba260bc40d not found: ID does not exist"
Oct 31 01:14:56.434777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.434755    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"]
Oct 31 01:14:56.442438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.442372    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-7f6d1e-599d949dcf-jswlb"]
Oct 31 01:14:56.457571 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.457536    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"]
Oct 31 01:14:56.461885 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.461860    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-acdaed/premium-w0-acdaed-kserve-869b8dbddd-ds6g9"]
Oct 31 01:14:56.470551 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.470496    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"]
Oct 31 01:14:56.476148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.476124    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-7f6d1e-85b975779-lpkps"]
Oct 31 01:14:56.498768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.498738    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wqng\" (UniqueName: \"kubernetes.io/projected/957a9635-2910-4b1b-9cc9-87e7672971f5-kube-api-access-2wqng\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.498768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.498764    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957a9635-2910-4b1b-9cc9-87e7672971f5-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:14:56.752377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.752304    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:14:56.752377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:56.752356    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:57.183241 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.183213    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" path="/var/lib/kubelet/pods/25a79474-3747-4ba2-b31d-754962b8f9e1/volumes"
Oct 31 01:14:57.183627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.183579    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" path="/var/lib/kubelet/pods/5b8b9992-fca7-49f1-b464-b445d8054d5d/volumes"
Oct 31 01:14:57.183983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.183967    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad74513f-78eb-4047-9421-79740761bbd6" path="/var/lib/kubelet/pods/ad74513f-78eb-4047-9421-79740761bbd6/volumes"
Oct 31 01:14:57.184670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.184655    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" path="/var/lib/kubelet/pods/bafc302a-4d33-44df-b0b2-29fdb44a84b8/volumes"
Oct 31 01:14:57.266090 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.266048    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:14:57.266254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.266109    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:57.369763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.369737    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"
Oct 31 01:14:57.369932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.369775    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc" event={"ID":"957a9635-2910-4b1b-9cc9-87e7672971f5","Type":"ContainerDied","Data":"a62f855cb3fe743b11083424c2bf73666e3fbaef967a0273fa1075bb45b6357b"}
Oct 31 01:14:57.369932 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.369836    8029 scope.go:122] "RemoveContainer" containerID="b728f0bd2e62b70118895f063017471259f54bc6838c1841c583dd8c78a7e90e"
Oct 31 01:14:57.391189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.391161    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"]
Oct 31 01:14:57.395464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:57.395445    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-28ff08-69d5f5f795-6srzc"]
Oct 31 01:14:58.681447 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.681415    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"]
Oct 31 01:14:58.682204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682181    8029 cpu_manager.go:414] "containerMap: removing container" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main"
Oct 31 01:14:58.682265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682209    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main"
Oct 31 01:14:58.682265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682227    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy"
Oct 31 01:14:58.682265 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682235    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682263    8029 cpu_manager.go:414] "containerMap: removing container" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682273    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682285    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerName="payload-processing"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682292    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerName="payload-processing"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682305    8029 cpu_manager.go:414] "containerMap: removing container" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerName="payload-pre-processing"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682312    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerName="payload-pre-processing"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682325    8029 cpu_manager.go:414] "containerMap: removing container" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy"
Oct 31 01:14:58.682355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682332    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682362    8029 cpu_manager.go:414] "containerMap: removing container" podUID="cec2b33b-3e06-41a9-9c6d-a1a4e1604368" containerName="revoke-keys"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682369    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cec2b33b-3e06-41a9-9c6d-a1a4e1604368" containerName="revoke-keys"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682378    8029 cpu_manager.go:414] "containerMap: removing container" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682385    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682395    8029 cpu_manager.go:414] "containerMap: removing container" podUID="88a225ed-f649-47bf-8496-2ad4544dfd43" containerName="revoke-keys"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682402    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88a225ed-f649-47bf-8496-2ad4544dfd43" containerName="revoke-keys"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682420    8029 cpu_manager.go:414] "containerMap: removing container" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerName="payload-pre-processing"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682426    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerName="payload-pre-processing"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682436    8029 cpu_manager.go:414] "containerMap: removing container" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682442    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682577    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="cec2b33b-3e06-41a9-9c6d-a1a4e1604368" containerName="revoke-keys"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682587    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d23822fb-a22c-4abe-8fb3-abe2879cdef9" containerName="istio-proxy"
Oct 31 01:14:58.682591 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682623    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="88a225ed-f649-47bf-8496-2ad4544dfd43" containerName="revoke-keys"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682638    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="bafc302a-4d33-44df-b0b2-29fdb44a84b8" containerName="payload-pre-processing"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682649    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682655    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a79474-3747-4ba2-b31d-754962b8f9e1" containerName="main"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682668    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad74513f-78eb-4047-9421-79740761bbd6" containerName="main"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682678    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="93013175-f94f-4939-8d9d-78930d529213" containerName="istio-proxy"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682688    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b8b9992-fca7-49f1-b464-b445d8054d5d" containerName="payload-processing"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682695    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" containerName="payload-pre-processing"
Oct 31 01:14:58.683002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.682704    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2813b0a-d570-425a-9523-778af9495993" containerName="istio-proxy"
Oct 31 01:14:58.688825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.688802    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.691927 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.691901    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:14:58.692524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.692507    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:14:58.692720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.692705    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:14:58.692803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.692786    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:14:58.693914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.693888    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"]
Oct 31 01:14:58.728669 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.728643    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plztl\" (UniqueName: \"kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.728902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.728881    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.729031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.729014    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.830043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.830014    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.830240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.830055    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plztl\" (UniqueName: \"kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.830240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.830189    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.830883 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.830861    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.832455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.832434    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:58.837955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:58.837930    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plztl\" (UniqueName: \"kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl\") pod \"maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:59.006775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:59.006699    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:14:59.184283 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:59.184253    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="957a9635-2910-4b1b-9cc9-87e7672971f5" path="/var/lib/kubelet/pods/957a9635-2910-4b1b-9cc9-87e7672971f5/volumes"
Oct 31 01:14:59.292016 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:14:59.291975    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e0b856_1a6a_45bf_9c4b_e0b1c6f2f3ae.slice/crio-c9a25f16601f8be09ea2cb22a271b3e5c32b3d6beaffb5d1cd2d3572217e978b WatchSource:0}: Error finding container c9a25f16601f8be09ea2cb22a271b3e5c32b3d6beaffb5d1cd2d3572217e978b: Status 404 returned error can't find the container with id c9a25f16601f8be09ea2cb22a271b3e5c32b3d6beaffb5d1cd2d3572217e978b
Oct 31 01:14:59.292518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:59.292482    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"]
Oct 31 01:14:59.347384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:59.347351    8029 patch_prober.go:29] interesting pod/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:14:59.347481 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:59.347406    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:14:59.399509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:14:59.399473    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx" event={"ID":"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae","Type":"ContainerStarted","Data":"c9a25f16601f8be09ea2cb22a271b3e5c32b3d6beaffb5d1cd2d3572217e978b"}
Oct 31 01:15:00.418800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.418770    8029 generic.go:356] "Generic (PLEG): container finished" podID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerID="767f37acc4134f0758a740ea3ec64a4d2dc223f8c4984af5525b8045882fdf06" exitCode=0
Oct 31 01:15:00.419261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.418840    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" event={"ID":"5cb2044d-95c3-409b-862d-d4758727f4bb","Type":"ContainerDied","Data":"767f37acc4134f0758a740ea3ec64a4d2dc223f8c4984af5525b8045882fdf06"}
Oct 31 01:15:00.419261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.418965    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr" event={"ID":"5cb2044d-95c3-409b-862d-d4758727f4bb","Type":"ContainerDied","Data":"9c524aa8bb8bd14e1c8b757c46f9354259a7c93b722b9e74ace7c5eba01955ea"}
Oct 31 01:15:00.419261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.419006    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c524aa8bb8bd14e1c8b757c46f9354259a7c93b722b9e74ace7c5eba01955ea"
Oct 31 01:15:00.421411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.421289    8029 generic.go:356] "Generic (PLEG): container finished" podID="26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" containerID="a2065de1bde97bcc31a85bcd171ebcd657bbe8fcabe30fab703414541bfdb3e3" exitCode=0
Oct 31 01:15:00.421411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.421377    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx" event={"ID":"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae","Type":"ContainerDied","Data":"a2065de1bde97bcc31a85bcd171ebcd657bbe8fcabe30fab703414541bfdb3e3"}
Oct 31 01:15:00.422407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.422386    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:15:00.429562 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.429513    8029 generic.go:356] "Generic (PLEG): container finished" podID="5303487b-1577-4432-b50d-a06e79249525" containerID="f7705e9036e1e00bb902d4234c576470d6bdc73483ea0127860fb772664c48ba" exitCode=0
Oct 31 01:15:00.429850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.429804    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" event={"ID":"5303487b-1577-4432-b50d-a06e79249525","Type":"ContainerDied","Data":"f7705e9036e1e00bb902d4234c576470d6bdc73483ea0127860fb772664c48ba"}
Oct 31 01:15:00.430026 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.429976    8029 scope.go:122] "RemoveContainer" containerID="093334fc610b9cd5eb75f8f76bf74024928a987870121206cb5ffe8d424e03bb"
Oct 31 01:15:00.438061 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.437756    8029 generic.go:356] "Generic (PLEG): container finished" podID="f19c3781-1aad-4392-9924-4a084cde6faf" containerID="4ad5b72a515e25c3ef76e605fdad322bcb1ac4d52b1568ca137895dc26c0f5d9" exitCode=0
Oct 31 01:15:00.438061 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.437988    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" event={"ID":"f19c3781-1aad-4392-9924-4a084cde6faf","Type":"ContainerDied","Data":"4ad5b72a515e25c3ef76e605fdad322bcb1ac4d52b1568ca137895dc26c0f5d9"}
Oct 31 01:15:00.451204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.451178    8029 generic.go:356] "Generic (PLEG): container finished" podID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerID="9a887b9d2170ef5204f0c57358f1bc1ddab4e6d1562ef268cbd2c1269ba7295c" exitCode=0
Oct 31 01:15:00.451375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.451355    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" event={"ID":"328e2256-e817-4994-90c8-4c27a7c1f5ab","Type":"ContainerDied","Data":"9a887b9d2170ef5204f0c57358f1bc1ddab4e6d1562ef268cbd2c1269ba7295c"}
Oct 31 01:15:00.451917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.451897    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.452030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.451964    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.452030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.451997    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.452141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.452054    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.452141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.452104    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.452234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.452192    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.452743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.452690    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.452840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.452787    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.453016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.452987    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.453091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453062    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.453164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453097    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.453164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453129    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert\") pod \"5cb2044d-95c3-409b-862d-d4758727f4bb\" (UID: \"5cb2044d-95c3-409b-862d-d4758727f4bb\") "
Oct 31 01:15:00.453502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453479    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.453803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453776    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:00.454051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453867    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.454051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453929    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.454051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453945    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.454051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.453959    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.477430 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.477405    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:15:00.487054 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.486451    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.487054 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.486920    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:00.487233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.487161    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "kube-api-access-msq5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.487393 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.487358    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token" pod "5cb2044d-95c3-409b-862d-d4758727f4bb" (UID: "5cb2044d-95c3-409b-862d-d4758727f4bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.508318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.506524    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:15:00.509200 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.509177    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:15:00.554618 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554570    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.554773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554652    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.554773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554671    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.554773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554690    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.554773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554713    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.554773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554743    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554781    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554813    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554862    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554868    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554901    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554956    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.554985    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.555013    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.555016    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:00.555056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.555045    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.555547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.555092    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:00.555547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.555439    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.556583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.555877    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.556583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.556116    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:00.556583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.556302    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.556869    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557090    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557124    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557155    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557195    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557235    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557290    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557325    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo\") pod \"328e2256-e817-4994-90c8-4c27a7c1f5ab\" (UID: \"328e2256-e817-4994-90c8-4c27a7c1f5ab\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557359    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557475    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557537    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data\") pod \"5303487b-1577-4432-b50d-a06e79249525\" (UID: \"5303487b-1577-4432-b50d-a06e79249525\") "
Oct 31 01:15:00.557663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.557572    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.559053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.558313    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.558542    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.558785    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.558798    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559053 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.558876    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.559142    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.559154    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.559392    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.559472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.559422    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.560386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.559716    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h\") pod \"f19c3781-1aad-4392-9924-4a084cde6faf\" (UID: \"f19c3781-1aad-4392-9924-4a084cde6faf\") "
Oct 31 01:15:00.560386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560353    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "kube-api-access-zmtrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560647    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560674    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msq5x\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-kube-api-access-msq5x\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560689    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560705    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560720    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560733    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f19c3781-1aad-4392-9924-4a084cde6faf-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560747    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5303487b-1577-4432-b50d-a06e79249525-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560760    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmtrj\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-kube-api-access-zmtrj\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560776    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560790    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560805    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb2044d-95c3-409b-862d-d4758727f4bb-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560819    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560833    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560845    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560847    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560858    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/328e2256-e817-4994-90c8-4c27a7c1f5ab-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560876    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560891    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560905    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560921    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560935    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb2044d-95c3-409b-862d-d4758727f4bb-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560948    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.560983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560960    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.562138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560901    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.562138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560949    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.562138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.561113    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.562138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.561685    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:00.562138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.562125    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo" pod "328e2256-e817-4994-90c8-4c27a7c1f5ab" (UID: "328e2256-e817-4994-90c8-4c27a7c1f5ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:00.562316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.561695    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:00.562316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.560963    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "kube-api-access-gpc8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.562436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.562419    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo" pod "5303487b-1577-4432-b50d-a06e79249525" (UID: "5303487b-1577-4432-b50d-a06e79249525"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:00.562592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.562572    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h" pod "f19c3781-1aad-4392-9924-4a084cde6faf" (UID: "f19c3781-1aad-4392-9924-4a084cde6faf"). InnerVolumeSpecName "kube-api-access-jwq4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:00.565914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.565892    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:15:00.588292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.588265    8029 patch_prober.go:29] interesting pod/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:15:00.588424 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.588321    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:15:00.663362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663274    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f19c3781-1aad-4392-9924-4a084cde6faf-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663303    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwq4h\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-kube-api-access-jwq4h\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663320    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663333    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f19c3781-1aad-4392-9924-4a084cde6faf-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663345    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpc8p\" (UniqueName: \"kubernetes.io/projected/5303487b-1577-4432-b50d-a06e79249525-kube-api-access-gpc8p\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663357    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5303487b-1577-4432-b50d-a06e79249525-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663370    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663382    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f19c3781-1aad-4392-9924-4a084cde6faf-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663395    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5303487b-1577-4432-b50d-a06e79249525-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:00.663687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:00.663408    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/328e2256-e817-4994-90c8-4c27a7c1f5ab-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.399104 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.399078    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:15:01.401435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.401407    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:15:01.456535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.456503    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:15:01.484801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.484714    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h" event={"ID":"f19c3781-1aad-4392-9924-4a084cde6faf","Type":"ContainerDied","Data":"102617539f49e12b64a7d96c2b1d6764549fc37c666b79d5550d5b12851aee4a"}
Oct 31 01:15:01.485034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.484862    8029 scope.go:122] "RemoveContainer" containerID="4ad5b72a515e25c3ef76e605fdad322bcb1ac4d52b1568ca137895dc26c0f5d9"
Oct 31 01:15:01.485251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.485222    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492314    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492409    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492438    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492472    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492512    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492576    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492720    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492774    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492802    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492850    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492938    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.492982    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493015    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493044    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493074    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493115    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493143    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket\") pod \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\" (UID: \"83bf61af-84f3-4b99-865d-e1cbd5a1918e\") "
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493196    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket\") pod \"8f818a5e-0da0-48b8-8a82-4b93068c7232\" (UID: \"8f818a5e-0da0-48b8-8a82-4b93068c7232\") "
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493545    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493755    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493929    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.493969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.493950    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.494355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.494183    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.501684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.501491    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:01.501913 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.501885    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.502041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.501928    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.502205 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.502169    8029 generic.go:356] "Generic (PLEG): container finished" podID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerID="9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053" exitCode=0
Oct 31 01:15:01.502429 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.502398    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" event={"ID":"2fcae1e4-b7a3-4677-807e-5a8a87bbe612","Type":"ContainerDied","Data":"9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053"}
Oct 31 01:15:01.502540 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.502457    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69" event={"ID":"2fcae1e4-b7a3-4677-807e-5a8a87bbe612","Type":"ContainerDied","Data":"60cb5c110dffdf71c96fad1b2792462f906057be61ae76c8e989fbe81f741e71"}
Oct 31 01:15:01.502540 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.502450    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.502700 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.502548    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"
Oct 31 01:15:01.502928 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.502897    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.503773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.503653    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.506843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.506446    8029 generic.go:356] "Generic (PLEG): container finished" podID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerID="1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1" exitCode=0
Oct 31 01:15:01.506843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.506517    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" event={"ID":"83bf61af-84f3-4b99-865d-e1cbd5a1918e","Type":"ContainerDied","Data":"1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1"}
Oct 31 01:15:01.506843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.506557    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd" event={"ID":"83bf61af-84f3-4b99-865d-e1cbd5a1918e","Type":"ContainerDied","Data":"f0c4d4a5cb385e161fc2552305e4f25a1b63e8661eff341c585d5caaafbf8727"}
Oct 31 01:15:01.506843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.506645    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"
Oct 31 01:15:01.506843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.506697    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.506843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.506776    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.511643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.509114    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.511643 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.509385    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "kube-api-access-f6lc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.512153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.512121    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:01.512338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.512309    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.512819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.512800    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:01.513753 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.513734    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt" pod "8f818a5e-0da0-48b8-8a82-4b93068c7232" (UID: "8f818a5e-0da0-48b8-8a82-4b93068c7232"). InnerVolumeSpecName "kube-api-access-tj9kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.515660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.514832    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token" pod "83bf61af-84f3-4b99-865d-e1cbd5a1918e" (UID: "83bf61af-84f3-4b99-865d-e1cbd5a1918e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.516924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.516901    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"
Oct 31 01:15:01.516995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.516956    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb" event={"ID":"328e2256-e817-4994-90c8-4c27a7c1f5ab","Type":"ContainerDied","Data":"84c66f4f4b2660c8038052d7a9ba9cc4ba442cd1651e1901f9f55bb27ed13041"}
Oct 31 01:15:01.517970 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.517946    8029 scope.go:122] "RemoveContainer" containerID="9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053"
Oct 31 01:15:01.528592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.528564    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"]
Oct 31 01:15:01.533634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.530226    8029 generic.go:356] "Generic (PLEG): container finished" podID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerID="ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411" exitCode=0
Oct 31 01:15:01.533634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.530384    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" event={"ID":"8f818a5e-0da0-48b8-8a82-4b93068c7232","Type":"ContainerDied","Data":"ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411"}
Oct 31 01:15:01.533634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.530427    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr" event={"ID":"8f818a5e-0da0-48b8-8a82-4b93068c7232","Type":"ContainerDied","Data":"57864a43b2f45c9eebd1167d47f44724d2ea910709477516a19e31ca2e288043"}
Oct 31 01:15:01.533634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.530507    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"
Oct 31 01:15:01.537342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.535881    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"
Oct 31 01:15:01.537342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.536512    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"
Oct 31 01:15:01.537342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.537038    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6" event={"ID":"5303487b-1577-4432-b50d-a06e79249525","Type":"ContainerDied","Data":"35b95912c66954bf5c5643dae559a460be240f6143461df2cd2263a76ddf1f14"}
Oct 31 01:15:01.550560 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.550538    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-qgr6h"]
Oct 31 01:15:01.551070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.551053    8029 scope.go:122] "RemoveContainer" containerID="9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053"
Oct 31 01:15:01.551419 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:01.551395    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053\": container with ID starting with 9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053 not found: ID does not exist" containerID="9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053"
Oct 31 01:15:01.551479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.551432    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053"} err="failed to get container status \"9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053\": rpc error: code = NotFound desc = could not find container \"9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053\": container with ID starting with 9dc337811f0e366f912677d2fcc4e3823b8f98b357b6b96ba24b477b98ad4053 not found: ID does not exist"
Oct 31 01:15:01.551479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.551458    8029 scope.go:122] "RemoveContainer" containerID="1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1"
Oct 31 01:15:01.564692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.564670    8029 scope.go:122] "RemoveContainer" containerID="64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"
Oct 31 01:15:01.573557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.573534    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"]
Oct 31 01:15:01.579590 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.579566    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-xpdsb"]
Oct 31 01:15:01.583790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.583738    8029 scope.go:122] "RemoveContainer" containerID="1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1"
Oct 31 01:15:01.584134 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:01.584111    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1\": container with ID starting with 1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1 not found: ID does not exist" containerID="1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1"
Oct 31 01:15:01.584223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.584139    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1"} err="failed to get container status \"1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1\": rpc error: code = NotFound desc = could not find container \"1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1\": container with ID starting with 1e4ff3f24772a6f6a49c1c793033f60e21c0e5689bb7091b7cdb1901075350d1 not found: ID does not exist"
Oct 31 01:15:01.584223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.584162    8029 scope.go:122] "RemoveContainer" containerID="64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"
Oct 31 01:15:01.588842 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:01.587109    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378\": container with ID starting with 64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378 not found: ID does not exist" containerID="64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"
Oct 31 01:15:01.588842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.587132    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378"} err="failed to get container status \"64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378\": rpc error: code = NotFound desc = could not find container \"64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378\": container with ID starting with 64e7d984bc388949587a835b7372aeaeb5fd925c082fe86032546c9c98ce0378 not found: ID does not exist"
Oct 31 01:15:01.588842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.587147    8029 scope.go:122] "RemoveContainer" containerID="9a887b9d2170ef5204f0c57358f1bc1ddab4e6d1562ef268cbd2c1269ba7295c"
Oct 31 01:15:01.594880 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.594856    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"]
Oct 31 01:15:01.595261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595240    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595294    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595412 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595360    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595415    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595521 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595473    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595574 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595523    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595578    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.595655 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595591    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595700    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595789    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.595900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595879    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert\") pod \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\" (UID: \"2fcae1e4-b7a3-4677-807e-5a8a87bbe612\") "
Oct 31 01:15:01.595962 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.595906    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.596032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596019    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.596458 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596433    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f818a5e-0da0-48b8-8a82-4b93068c7232-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.596535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596464    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.596535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596480    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.596535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596495    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.596535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596508    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.596535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596521    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.596535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596533    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596547    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596560    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596572    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596586    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596616    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj9kt\" (UniqueName: \"kubernetes.io/projected/8f818a5e-0da0-48b8-8a82-4b93068c7232-kube-api-access-tj9kt\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596631    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596643    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596656    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596670    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6lc7\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-kube-api-access-f6lc7\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596683    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596696    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f818a5e-0da0-48b8-8a82-4b93068c7232-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596709    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.597486 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.596722    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf61af-84f3-4b99-865d-e1cbd5a1918e-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.598197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.597502    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:01.602101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.601171    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:01.603915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.603820    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "kube-api-access-xd92b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.609562 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.609529    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:01.610863 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.610841    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token" pod "2fcae1e4-b7a3-4677-807e-5a8a87bbe612" (UID: "2fcae1e4-b7a3-4677-807e-5a8a87bbe612"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.611562 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.611546    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-dqwnr"]
Oct 31 01:15:01.615666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.615645    8029 scope.go:122] "RemoveContainer" containerID="ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411"
Oct 31 01:15:01.619121 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.619105    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"]
Oct 31 01:15:01.626906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.626858    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-gzcmr"]
Oct 31 01:15:01.632036 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.632018    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"]
Oct 31 01:15:01.635798 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.635778    8029 scope.go:122] "RemoveContainer" containerID="ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411"
Oct 31 01:15:01.636637 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:01.636461    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411\": container with ID starting with ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411 not found: ID does not exist" containerID="ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411"
Oct 31 01:15:01.636637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.636494    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411"} err="failed to get container status \"ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411\": rpc error: code = NotFound desc = could not find container \"ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411\": container with ID starting with ab5f68035fb3ec1e9371525eb752fb1a72bb092f2bdabba0c712641efad12411 not found: ID does not exist"
Oct 31 01:15:01.636637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.636519    8029 scope.go:122] "RemoveContainer" containerID="f7705e9036e1e00bb902d4234c576470d6bdc73483ea0127860fb772664c48ba"
Oct 31 01:15:01.640452 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.640431    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7f6d1e-openshift-default-67d6894945-9llj6"]
Oct 31 01:15:01.697629 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.697585    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.697748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.697631    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.697748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.697646    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd92b\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-kube-api-access-xd92b\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.697748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.697657    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.697748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.697666    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fcae1e4-b7a3-4677-807e-5a8a87bbe612-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.711189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.711167    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:15:01.798097 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.798070    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl\" (UniqueName: \"kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl\") pod \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") "
Oct 31 01:15:01.798255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.798244    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca\") pod \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") "
Oct 31 01:15:01.798320 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.798287    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca\") pod \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\" (UID: \"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae\") "
Oct 31 01:15:01.798661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.798627    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca" pod "26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" (UID: "26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:01.800358 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.800335    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl" pod "26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" (UID: "26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae"). InnerVolumeSpecName "kube-api-access-plztl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:01.800481 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.800462    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca" pod "26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" (UID: "26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:15:01.887972 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.871528    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"]
Oct 31 01:15:01.894751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.894712    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-68h69"]
Oct 31 01:15:01.899854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.899824    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"]
Oct 31 01:15:01.901106 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.901033    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.901246 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.901231    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plztl\" (UniqueName: \"kubernetes.io/projected/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-kube-api-access-plztl\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.901353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.901341    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:01.911343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:01.911318    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-28ff08-openshift-default-7c95bbc97f-gc2wd"]
Oct 31 01:15:02.564126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:02.564097    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"
Oct 31 01:15:02.564813 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:02.564132    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx" event={"ID":"26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae","Type":"ContainerDied","Data":"c9a25f16601f8be09ea2cb22a271b3e5c32b3d6beaffb5d1cd2d3572217e978b"}
Oct 31 01:15:02.564950 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:02.564843    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9a25f16601f8be09ea2cb22a271b3e5c32b3d6beaffb5d1cd2d3572217e978b"
Oct 31 01:15:03.189592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.189558    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" path="/var/lib/kubelet/pods/2fcae1e4-b7a3-4677-807e-5a8a87bbe612/volumes"
Oct 31 01:15:03.190750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.190723    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" path="/var/lib/kubelet/pods/328e2256-e817-4994-90c8-4c27a7c1f5ab/volumes"
Oct 31 01:15:03.191449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.191426    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5303487b-1577-4432-b50d-a06e79249525" path="/var/lib/kubelet/pods/5303487b-1577-4432-b50d-a06e79249525/volumes"
Oct 31 01:15:03.191902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.191887    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" path="/var/lib/kubelet/pods/5cb2044d-95c3-409b-862d-d4758727f4bb/volumes"
Oct 31 01:15:03.192687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.192639    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" path="/var/lib/kubelet/pods/83bf61af-84f3-4b99-865d-e1cbd5a1918e/volumes"
Oct 31 01:15:03.193073 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.193053    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" path="/var/lib/kubelet/pods/8f818a5e-0da0-48b8-8a82-4b93068c7232/volumes"
Oct 31 01:15:03.193470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.193458    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" path="/var/lib/kubelet/pods/f19c3781-1aad-4392-9924-4a084cde6faf/volumes"
Oct 31 01:15:03.481701 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:03.481613    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"]
Oct 31 01:15:06.854664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.854549    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"]
Oct 31 01:15:06.855479 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.855428    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy" containerID="cri-o://fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199" gracePeriod=30
Oct 31 01:15:06.879878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.879848    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"]
Oct 31 01:15:06.880083 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.880062    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" containerID="cri-o://d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969" gracePeriod=30
Oct 31 01:15:06.887731 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.887695    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"]
Oct 31 01:15:06.889319 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.888127    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy" containerID="cri-o://c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f" gracePeriod=30
Oct 31 01:15:06.893464 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.893436    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"]
Oct 31 01:15:06.893804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:06.893772    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" containerID="cri-o://b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce" gracePeriod=30
Oct 31 01:15:07.388470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:07.388428    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:15:07.388693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:07.388494    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:15:08.338644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:08.337530    8029 patch_prober.go:29] interesting pod/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:15:08.338644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:08.337588    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:15:08.823772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:08.823739    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"]
Oct 31 01:15:09.650149 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650106    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"]
Oct 31 01:15:09.650681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650665    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650681    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650692    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650697    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650713    8029 cpu_manager.go:414] "containerMap: removing container" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650718    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650726    8029 cpu_manager.go:414] "containerMap: removing container" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650731    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650742    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650746    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy"
Oct 31 01:15:09.650751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650755    8029 cpu_manager.go:414] "containerMap: removing container" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650760    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650773    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650777    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650784    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650788    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650794    8029 cpu_manager.go:414] "containerMap: removing container" podUID="26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" containerName="revoke-keys"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650798    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" containerName="revoke-keys"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650817    8029 cpu_manager.go:414] "containerMap: removing container" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650821    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650901    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650909    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcfa0bd8-3aa9-4fc7-91a1-a1c0b4c8ff92" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650916    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fcae1e4-b7a3-4677-807e-5a8a87bbe612" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650923    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f19c3781-1aad-4392-9924-4a084cde6faf" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650929    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650935    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="328e2256-e817-4994-90c8-4c27a7c1f5ab" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650942    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5303487b-1577-4432-b50d-a06e79249525" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650948    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5cb2044d-95c3-409b-862d-d4758727f4bb" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650957    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" containerName="revoke-keys"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650966    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy"
Oct 31 01:15:09.651071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.650976    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f818a5e-0da0-48b8-8a82-4b93068c7232" containerName="istio-proxy"
Oct 31 01:15:09.656578 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.656550    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.661866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.661158    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:15:09.661866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.661207    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:15:09.662351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.662240    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"]
Oct 31 01:15:09.662964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.662941    8029 cpu_manager.go:414] "containerMap: removing container" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy"
Oct 31 01:15:09.662964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.662964    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83bf61af-84f3-4b99-865d-e1cbd5a1918e" containerName="istio-proxy"
Oct 31 01:15:09.664969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.664939    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:15:09.665089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.664967    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:15:09.682274 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.674494    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"]
Oct 31 01:15:09.682274 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.674634    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.682274 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.676272    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"]
Oct 31 01:15:09.715769 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.715735    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.715913 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.715782    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn9dw\" (UniqueName: \"kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.715913 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.715899    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.817439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.817407    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.817631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.817513    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.817631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.817564    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn9dw\" (UniqueName: \"kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.817747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.817713    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2pxs\" (UniqueName: \"kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.817797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.817768    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.817895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.817877    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.818150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.818133    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.820343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.820323    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.825192 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.825173    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn9dw\" (UniqueName: \"kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw\") pod \"maas-api-revoke-sub-771a8d91-kz5c5\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.919317 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.919237    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2pxs\" (UniqueName: \"kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.919317 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.919298    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.919487 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.919367    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.919948 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.919923    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.921825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.921806    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.927299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.927277    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2pxs\" (UniqueName: \"kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs\") pod \"maas-api-revoke-sub-74995c3b-wr4fn\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:09.982758 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.982731    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:09.993024 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:09.993006    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:10.266889 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:15:10.266851    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dab6a92_5447_4100_a239_e35458f07490.slice/crio-8d009b486429c1b3f6a1b2a873d5ae58aa37ee1c4f72e53a793ea092ecfbe7ad WatchSource:0}: Error finding container 8d009b486429c1b3f6a1b2a873d5ae58aa37ee1c4f72e53a793ea092ecfbe7ad: Status 404 returned error can't find the container with id 8d009b486429c1b3f6a1b2a873d5ae58aa37ee1c4f72e53a793ea092ecfbe7ad
Oct 31 01:15:10.267045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.267008    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"]
Oct 31 01:15:10.280193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.280170    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"]
Oct 31 01:15:10.283030 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:15:10.283007    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bd58bf_c212_474b_b637_10cc13e086fe.slice/crio-4aa6c619b214031160f21866e1e03be11e97ee4223b8d92284dd73370115626f WatchSource:0}: Error finding container 4aa6c619b214031160f21866e1e03be11e97ee4223b8d92284dd73370115626f: Status 404 returned error can't find the container with id 4aa6c619b214031160f21866e1e03be11e97ee4223b8d92284dd73370115626f
Oct 31 01:15:10.567076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.566994    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:15:10.634912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.634880    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn" event={"ID":"63bd58bf-c212-474b-b637-10cc13e086fe","Type":"ContainerStarted","Data":"af5221870324eff01a072beab270915e4fd216533b5dcbf2df6f8fcacd4990db"}
Oct 31 01:15:10.635019 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.634955    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn" event={"ID":"63bd58bf-c212-474b-b637-10cc13e086fe","Type":"ContainerStarted","Data":"4aa6c619b214031160f21866e1e03be11e97ee4223b8d92284dd73370115626f"}
Oct 31 01:15:10.636627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.636580    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5" event={"ID":"8dab6a92-5447-4100-a239-e35458f07490","Type":"ContainerStarted","Data":"389265b61dc346afa7f001da822ac0c828ade0ac38f81aa6ca216d67a445426f"}
Oct 31 01:15:10.636741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.636650    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5" event={"ID":"8dab6a92-5447-4100-a239-e35458f07490","Type":"ContainerStarted","Data":"8d009b486429c1b3f6a1b2a873d5ae58aa37ee1c4f72e53a793ea092ecfbe7ad"}
Oct 31 01:15:10.655654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.655586    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn" podStartSLOduration=1.655569412 podStartE2EDuration="1.655569412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:15:10.651120437 +0000 UTC m=+1107.638232414" watchObservedRunningTime="2026-10-31 01:15:10.655569412 +0000 UTC m=+1107.642681391"
Oct 31 01:15:10.666368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:10.666316    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5" podStartSLOduration=1.6662999269999998 podStartE2EDuration="1.666299927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:15:10.665640508 +0000 UTC m=+1107.652752485" watchObservedRunningTime="2026-10-31 01:15:10.666299927 +0000 UTC m=+1107.653411906"
Oct 31 01:15:11.588222 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.588196    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:15:11.639268 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.639230    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume\") pod \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") "
Oct 31 01:15:11.639457 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.639400    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n\" (UniqueName: \"kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n\") pod \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\" (UID: \"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9\") "
Oct 31 01:15:11.639641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.639612    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume" pod "5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" (UID: "5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:11.639863 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.639840    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:11.641784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.641763    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n" pod "5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" (UID: "5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9"). InnerVolumeSpecName "kube-api-access-4qk7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:11.649210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.649185    8029 generic.go:356] "Generic (PLEG): container finished" podID="63bd58bf-c212-474b-b637-10cc13e086fe" containerID="af5221870324eff01a072beab270915e4fd216533b5dcbf2df6f8fcacd4990db" exitCode=0
Oct 31 01:15:11.649312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.649270    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn" event={"ID":"63bd58bf-c212-474b-b637-10cc13e086fe","Type":"ContainerDied","Data":"af5221870324eff01a072beab270915e4fd216533b5dcbf2df6f8fcacd4990db"}
Oct 31 01:15:11.650840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.650819    8029 generic.go:356] "Generic (PLEG): container finished" podID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerID="6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9" exitCode=0
Oct 31 01:15:11.650921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.650860    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"
Oct 31 01:15:11.650921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.650888    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" event={"ID":"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9","Type":"ContainerDied","Data":"6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9"}
Oct 31 01:15:11.651080 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.650927    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj" event={"ID":"5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9","Type":"ContainerDied","Data":"5d0b208093653346068508cdc179d25b6d55bb2ecb81527ac05584a1fa9f4398"}
Oct 31 01:15:11.651080 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.650948    8029 scope.go:122] "RemoveContainer" containerID="6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9"
Oct 31 01:15:11.652710 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.652687    8029 generic.go:356] "Generic (PLEG): container finished" podID="8dab6a92-5447-4100-a239-e35458f07490" containerID="389265b61dc346afa7f001da822ac0c828ade0ac38f81aa6ca216d67a445426f" exitCode=0
Oct 31 01:15:11.652824 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.652711    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5" event={"ID":"8dab6a92-5447-4100-a239-e35458f07490","Type":"ContainerDied","Data":"389265b61dc346afa7f001da822ac0c828ade0ac38f81aa6ca216d67a445426f"}
Oct 31 01:15:11.662667 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.662651    8029 scope.go:122] "RemoveContainer" containerID="6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9"
Oct 31 01:15:11.662954 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:11.662929    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9\": container with ID starting with 6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9 not found: ID does not exist" containerID="6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9"
Oct 31 01:15:11.662999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.662955    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9"} err="failed to get container status \"6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9\": rpc error: code = NotFound desc = could not find container \"6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9\": container with ID starting with 6c62d0c3d12e0a6521f7c449a067640b53d474c3c9bc82f88f27b747675cf1a9 not found: ID does not exist"
Oct 31 01:15:11.682000 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.681974    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"]
Oct 31 01:15:11.687531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.687485    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-acdaed-7b8f85ccf5-4drkj"]
Oct 31 01:15:11.741231 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:11.741195    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qk7n\" (UniqueName: \"kubernetes.io/projected/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9-kube-api-access-4qk7n\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.195748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.195721    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:15:12.234132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.234100    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:15:12.247040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247012    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.247224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247061    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.247224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247139    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.247224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247171    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.247389 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247288    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.247389 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247340    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.247502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247381    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.247502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247427    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.247502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247458    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.247502 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247493    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247547    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247581    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247630    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7\") pod \"87c94909-be59-4713-8244-3444fe3c8b0a\" (UID: \"87c94909-be59-4713-8244-3444fe3c8b0a\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247655    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247662    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247721    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247850    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247869    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247919    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.248197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.247951    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket\") pod \"67747552-c23c-449f-b7dd-968dbd0e1020\" (UID: \"67747552-c23c-449f-b7dd-968dbd0e1020\") "
Oct 31 01:15:12.249552 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.249394    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.249965 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.249710    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.250888    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.251285    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.251288    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.251504    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.251782    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253143    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253193    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253217    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253231    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253245    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253262    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253275    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253289    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253302    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87c94909-be59-4713-8244-3444fe3c8b0a-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.253514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.253316    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.254577 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.254012    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:12.255432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.255121    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.256763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.256727    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.256865 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.256801    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "kube-api-access-69cm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.257353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.257317    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:12.257496 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.257472    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token" pod "87c94909-be59-4713-8244-3444fe3c8b0a" (UID: "87c94909-be59-4713-8244-3444fe3c8b0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.258310 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.258286    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.259853 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.259816    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5" pod "67747552-c23c-449f-b7dd-968dbd0e1020" (UID: "67747552-c23c-449f-b7dd-968dbd0e1020"). InnerVolumeSpecName "kube-api-access-6s5k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.269911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.269887    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:15:12.271989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.271973    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:15:12.353638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353577    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.353835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353670    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.353835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353728    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.353835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353774    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.353835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353816    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.353835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353834    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353882    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353908    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353955    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.353993    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354031    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354063    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354095    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354118    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354127    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354197    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert\") pod \"05bb1ced-152a-488d-8b65-d70892db4c2e\" (UID: \"05bb1ced-152a-488d-8b65-d70892db4c2e\") "
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354250    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354304    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354305    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354348    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token\") pod \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\" (UID: \"e9658b33-0dd9-42b5-9e2f-4e406b19b988\") "
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354621    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354833    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354846    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354867    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87c94909-be59-4713-8244-3444fe3c8b0a-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354883    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354900    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87c94909-be59-4713-8244-3444fe3c8b0a-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.354944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354936    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67747552-c23c-449f-b7dd-968dbd0e1020-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354952    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69cm7\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-kube-api-access-69cm7\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354967    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67747552-c23c-449f-b7dd-968dbd0e1020-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354982    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s5k5\" (UniqueName: \"kubernetes.io/projected/67747552-c23c-449f-b7dd-968dbd0e1020-kube-api-access-6s5k5\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354994    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.354996    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67747552-c23c-449f-b7dd-968dbd0e1020-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.355020    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87c94909-be59-4713-8244-3444fe3c8b0a-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.355035    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.355052    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.355233    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.355312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.355296    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.356435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.356407    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:12.356615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.356578    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:12.356770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.356751    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.356851 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.356825    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "kube-api-access-525nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.357008 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.356989    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.357095 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.357076    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.358058 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.358031    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.358350 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.358330    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy" pod "05bb1ced-152a-488d-8b65-d70892db4c2e" (UID: "05bb1ced-152a-488d-8b65-d70892db4c2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.358885 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.358850    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.358999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.358977    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "kube-api-access-x855v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.359184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.359160    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy" pod "e9658b33-0dd9-42b5-9e2f-4e406b19b988" (UID: "e9658b33-0dd9-42b5-9e2f-4e406b19b988"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:12.456556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456525    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05bb1ced-152a-488d-8b65-d70892db4c2e-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456552    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x855v\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-kube-api-access-x855v\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456563    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456572    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456580    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-525nw\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-kube-api-access-525nw\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456589    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456626    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456635    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456643    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456651    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456659    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456667    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456674    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456684    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05bb1ced-152a-488d-8b65-d70892db4c2e-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.456766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.456697    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9658b33-0dd9-42b5-9e2f-4e406b19b988-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.663254 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.663223    8029 generic.go:356] "Generic (PLEG): container finished" podID="87c94909-be59-4713-8244-3444fe3c8b0a" containerID="b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce" exitCode=0
Oct 31 01:15:12.663717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.663283    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"
Oct 31 01:15:12.663717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.663307    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" event={"ID":"87c94909-be59-4713-8244-3444fe3c8b0a","Type":"ContainerDied","Data":"b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce"}
Oct 31 01:15:12.663717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.663394    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl" event={"ID":"87c94909-be59-4713-8244-3444fe3c8b0a","Type":"ContainerDied","Data":"e7b4b936c2e0e4e9a9e395a5bab41ef2e395650c6ab78c3196e133421c35b853"}
Oct 31 01:15:12.663717 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.663425    8029 scope.go:122] "RemoveContainer" containerID="b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce"
Oct 31 01:15:12.664943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.664920    8029 generic.go:356] "Generic (PLEG): container finished" podID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerID="c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f" exitCode=0
Oct 31 01:15:12.665070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.664982    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"
Oct 31 01:15:12.665441 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.665015    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" event={"ID":"05bb1ced-152a-488d-8b65-d70892db4c2e","Type":"ContainerDied","Data":"c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f"}
Oct 31 01:15:12.665513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.665472    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj" event={"ID":"05bb1ced-152a-488d-8b65-d70892db4c2e","Type":"ContainerDied","Data":"b66b1ec610e1e1a27470c92198367ad7472ce2d79b1940408e8b331f8a151697"}
Oct 31 01:15:12.667010 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.666987    8029 generic.go:356] "Generic (PLEG): container finished" podID="67747552-c23c-449f-b7dd-968dbd0e1020" containerID="fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199" exitCode=0
Oct 31 01:15:12.667111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.667058    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" event={"ID":"67747552-c23c-449f-b7dd-968dbd0e1020","Type":"ContainerDied","Data":"fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199"}
Oct 31 01:15:12.667174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.667120    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq" event={"ID":"67747552-c23c-449f-b7dd-968dbd0e1020","Type":"ContainerDied","Data":"cd4f1a359498d16b8b18881daedba3f1d64afe77cb240f62e36b3aad44b42a6f"}
Oct 31 01:15:12.667174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.667122    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"
Oct 31 01:15:12.669106 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.669085    8029 generic.go:356] "Generic (PLEG): container finished" podID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerID="d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969" exitCode=0
Oct 31 01:15:12.669201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.669113    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" event={"ID":"e9658b33-0dd9-42b5-9e2f-4e406b19b988","Type":"ContainerDied","Data":"d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969"}
Oct 31 01:15:12.669201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.669140    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"
Oct 31 01:15:12.669201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.669158    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr" event={"ID":"e9658b33-0dd9-42b5-9e2f-4e406b19b988","Type":"ContainerDied","Data":"bab41ca513891a6cbc0c49606713270bad1b20f84d9fbda052f0077d7fef3913"}
Oct 31 01:15:12.680198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.680179    8029 scope.go:122] "RemoveContainer" containerID="b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce"
Oct 31 01:15:12.680524 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:12.680490    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce\": container with ID starting with b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce not found: ID does not exist" containerID="b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce"
Oct 31 01:15:12.680660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.680526    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce"} err="failed to get container status \"b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce\": rpc error: code = NotFound desc = could not find container \"b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce\": container with ID starting with b2fc92ceb050e2b7c3e9f2d62a66712a2724eb4bb8cf78044c828540a8bbe2ce not found: ID does not exist"
Oct 31 01:15:12.680660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.680547    8029 scope.go:122] "RemoveContainer" containerID="c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f"
Oct 31 01:15:12.694995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.694973    8029 scope.go:122] "RemoveContainer" containerID="c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f"
Oct 31 01:15:12.695343 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:12.695315    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f\": container with ID starting with c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f not found: ID does not exist" containerID="c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f"
Oct 31 01:15:12.695440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.695355    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f"} err="failed to get container status \"c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f\": rpc error: code = NotFound desc = could not find container \"c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f\": container with ID starting with c26c24f37899f3ca9ea05b78deb3b5b7bb48f48de94e8d8993f512037614b97f not found: ID does not exist"
Oct 31 01:15:12.695440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.695380    8029 scope.go:122] "RemoveContainer" containerID="fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199"
Oct 31 01:15:12.697181 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.697148    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"]
Oct 31 01:15:12.702135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.702108    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-2mntl"]
Oct 31 01:15:12.712541 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.711351    8029 scope.go:122] "RemoveContainer" containerID="fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199"
Oct 31 01:15:12.712541 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:12.711752    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199\": container with ID starting with fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199 not found: ID does not exist" containerID="fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199"
Oct 31 01:15:12.712541 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.711779    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199"} err="failed to get container status \"fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199\": rpc error: code = NotFound desc = could not find container \"fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199\": container with ID starting with fddc36d0b7a5cebd5a5d4dd1cd3c388cca9a5570bfa2d8ba20089edbccb7e199 not found: ID does not exist"
Oct 31 01:15:12.712541 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.711800    8029 scope.go:122] "RemoveContainer" containerID="d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969"
Oct 31 01:15:12.716797 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.716777    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"]
Oct 31 01:15:12.724040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.724017    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4d2zr"]
Oct 31 01:15:12.726896 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.726864    8029 scope.go:122] "RemoveContainer" containerID="d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969"
Oct 31 01:15:12.727249 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:12.727210    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969\": container with ID starting with d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969 not found: ID does not exist" containerID="d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969"
Oct 31 01:15:12.727312 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.727260    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969"} err="failed to get container status \"d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969\": rpc error: code = NotFound desc = could not find container \"d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969\": container with ID starting with d144a61a35dcc302fb342321e7d6f332c09c4382f5d6ea8e86754e209b164969 not found: ID does not exist"
Oct 31 01:15:12.731420 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.731399    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"]
Oct 31 01:15:12.735349 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.735328    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-nhzxj"]
Oct 31 01:15:12.746475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.746435    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"]
Oct 31 01:15:12.752902 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.752876    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-acdaed-openshift-default-fd4ccfb57-4pxmq"]
Oct 31 01:15:12.821747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.821719    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:12.860952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.860920    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca\") pod \"8dab6a92-5447-4100-a239-e35458f07490\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") "
Oct 31 01:15:12.861114 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.861074    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca\") pod \"8dab6a92-5447-4100-a239-e35458f07490\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") "
Oct 31 01:15:12.861259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.861236    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw\" (UniqueName: \"kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw\") pod \"8dab6a92-5447-4100-a239-e35458f07490\" (UID: \"8dab6a92-5447-4100-a239-e35458f07490\") "
Oct 31 01:15:12.861656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.861590    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca" pod "8dab6a92-5447-4100-a239-e35458f07490" (UID: "8dab6a92-5447-4100-a239-e35458f07490"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.862255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.862234    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8dab6a92-5447-4100-a239-e35458f07490-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.863575 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.863554    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca" pod "8dab6a92-5447-4100-a239-e35458f07490" (UID: "8dab6a92-5447-4100-a239-e35458f07490"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:15:12.863800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.863783    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw" pod "8dab6a92-5447-4100-a239-e35458f07490" (UID: "8dab6a92-5447-4100-a239-e35458f07490"). InnerVolumeSpecName "kube-api-access-vn9dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.891908 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.891888    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:12.963715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.963626    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs\" (UniqueName: \"kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs\") pod \"63bd58bf-c212-474b-b637-10cc13e086fe\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") "
Oct 31 01:15:12.963879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.963736    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca\") pod \"63bd58bf-c212-474b-b637-10cc13e086fe\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") "
Oct 31 01:15:12.963879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.963835    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca\") pod \"63bd58bf-c212-474b-b637-10cc13e086fe\" (UID: \"63bd58bf-c212-474b-b637-10cc13e086fe\") "
Oct 31 01:15:12.964159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.964135    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca" pod "63bd58bf-c212-474b-b637-10cc13e086fe" (UID: "63bd58bf-c212-474b-b637-10cc13e086fe"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:12.964229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.964200    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn9dw\" (UniqueName: \"kubernetes.io/projected/8dab6a92-5447-4100-a239-e35458f07490-kube-api-access-vn9dw\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.964229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.964222    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8dab6a92-5447-4100-a239-e35458f07490-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:12.965789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.965763    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs" pod "63bd58bf-c212-474b-b637-10cc13e086fe" (UID: "63bd58bf-c212-474b-b637-10cc13e086fe"). InnerVolumeSpecName "kube-api-access-p2pxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:12.965900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:12.965863    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca" pod "63bd58bf-c212-474b-b637-10cc13e086fe" (UID: "63bd58bf-c212-474b-b637-10cc13e086fe"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:15:13.065386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.065351    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:13.065386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.065381    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2pxs\" (UniqueName: \"kubernetes.io/projected/63bd58bf-c212-474b-b637-10cc13e086fe-kube-api-access-p2pxs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:13.065386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.065392    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63bd58bf-c212-474b-b637-10cc13e086fe-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:13.183538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.183503    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" path="/var/lib/kubelet/pods/05bb1ced-152a-488d-8b65-d70892db4c2e/volumes"
Oct 31 01:15:13.184018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.183989    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" path="/var/lib/kubelet/pods/5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9/volumes"
Oct 31 01:15:13.184470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.184451    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" path="/var/lib/kubelet/pods/67747552-c23c-449f-b7dd-968dbd0e1020/volumes"
Oct 31 01:15:13.185262 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.185241    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" path="/var/lib/kubelet/pods/87c94909-be59-4713-8244-3444fe3c8b0a/volumes"
Oct 31 01:15:13.185781 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.185763    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" path="/var/lib/kubelet/pods/e9658b33-0dd9-42b5-9e2f-4e406b19b988/volumes"
Oct 31 01:15:13.680040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.680011    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"
Oct 31 01:15:13.680040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.680018    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5" event={"ID":"8dab6a92-5447-4100-a239-e35458f07490","Type":"ContainerDied","Data":"8d009b486429c1b3f6a1b2a873d5ae58aa37ee1c4f72e53a793ea092ecfbe7ad"}
Oct 31 01:15:13.680532 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.680065    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d009b486429c1b3f6a1b2a873d5ae58aa37ee1c4f72e53a793ea092ecfbe7ad"
Oct 31 01:15:13.686431 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.686395    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"
Oct 31 01:15:13.686652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.686485    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn" event={"ID":"63bd58bf-c212-474b-b637-10cc13e086fe","Type":"ContainerDied","Data":"4aa6c619b214031160f21866e1e03be11e97ee4223b8d92284dd73370115626f"}
Oct 31 01:15:13.686652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.686538    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4aa6c619b214031160f21866e1e03be11e97ee4223b8d92284dd73370115626f"
Oct 31 01:15:13.824887 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:13.824834    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" podUID="6da29a69-9b53-4506-8db0-4758c504af67" containerName="payload-pre-processing" containerID="cri-o://44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711" gracePeriod=25
Oct 31 01:15:18.503100 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.503057    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" containerID="cri-o://93a008dfe697355201a758ba5f7c886c3e6af1f50922473c02080672a73937a6" gracePeriod=45
Oct 31 01:15:18.645948 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.645907    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Oct 31 01:15:18.747023 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.746994    8029 generic.go:356] "Generic (PLEG): container finished" podID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerID="93a008dfe697355201a758ba5f7c886c3e6af1f50922473c02080672a73937a6" exitCode=0
Oct 31 01:15:18.747170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.747092    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" event={"ID":"3288e4f3-0914-457e-bb66-ba36c4c2edc8","Type":"ContainerDied","Data":"93a008dfe697355201a758ba5f7c886c3e6af1f50922473c02080672a73937a6"}
Oct 31 01:15:18.747170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.747150    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k" event={"ID":"3288e4f3-0914-457e-bb66-ba36c4c2edc8","Type":"ContainerDied","Data":"0440bbc07c7679f3c271471267ff25c8df7e38da08535c666955a1bd9b3c88e3"}
Oct 31 01:15:18.747342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.747162    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0440bbc07c7679f3c271471267ff25c8df7e38da08535c666955a1bd9b3c88e3"
Oct 31 01:15:18.761251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.761113    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:15:18.825616 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.825567    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home\") pod \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") "
Oct 31 01:15:18.825763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.825651    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv\" (UniqueName: \"kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv\") pod \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") "
Oct 31 01:15:18.825763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.825673    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache\") pod \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") "
Oct 31 01:15:18.825763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.825705    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir\") pod \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") "
Oct 31 01:15:18.825763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.825731    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm\") pod \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") "
Oct 31 01:15:18.825763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.825769    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs\" (UniqueName: \"kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs\") pod \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\" (UID: \"3288e4f3-0914-457e-bb66-ba36c4c2edc8\") "
Oct 31 01:15:18.829811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.826235    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home" pod "3288e4f3-0914-457e-bb66-ba36c4c2edc8" (UID: "3288e4f3-0914-457e-bb66-ba36c4c2edc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:18.829811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.826304    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir" pod "3288e4f3-0914-457e-bb66-ba36c4c2edc8" (UID: "3288e4f3-0914-457e-bb66-ba36c4c2edc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:18.829811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.826946    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache" pod "3288e4f3-0914-457e-bb66-ba36c4c2edc8" (UID: "3288e4f3-0914-457e-bb66-ba36c4c2edc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:18.829811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.829567    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm" pod "3288e4f3-0914-457e-bb66-ba36c4c2edc8" (UID: "3288e4f3-0914-457e-bb66-ba36c4c2edc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:18.829811 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.829726    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv" pod "3288e4f3-0914-457e-bb66-ba36c4c2edc8" (UID: "3288e4f3-0914-457e-bb66-ba36c4c2edc8"). InnerVolumeSpecName "kube-api-access-8zmcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:18.831029 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.830996    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs" pod "3288e4f3-0914-457e-bb66-ba36c4c2edc8" (UID: "3288e4f3-0914-457e-bb66-ba36c4c2edc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:15:18.928159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.928093    8029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-home\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:18.928159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.928123    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zmcv\" (UniqueName: \"kubernetes.io/projected/3288e4f3-0914-457e-bb66-ba36c4c2edc8-kube-api-access-8zmcv\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:18.928159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.928136    8029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-model-cache\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:18.928159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.928150    8029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tmp-dir\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:18.928159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.928161    8029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288e4f3-0914-457e-bb66-ba36c4c2edc8-dshm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:18.928159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:18.928172    8029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288e4f3-0914-457e-bb66-ba36c4c2edc8-tls-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:19.443275 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:19.443174    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:15:19.755121 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:19.755050    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"
Oct 31 01:15:19.772550 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:19.772523    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"]
Oct 31 01:15:19.779159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:19.779135    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-487a6c/simulator-w3-487a6c-kserve-5c95bf5f5f-k8h9k"]
Oct 31 01:15:20.565675 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:20.565640    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" podUID="6da29a69-9b53-4506-8db0-4758c504af67" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:15:21.182302 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:21.182273    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" path="/var/lib/kubelet/pods/3288e4f3-0914-457e-bb66-ba36c4c2edc8/volumes"
Oct 31 01:15:25.333740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:25.333708    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"]
Oct 31 01:15:25.339728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:25.339700    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-487a6c-sv9hx"]
Oct 31 01:15:27.182515 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:27.182482    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae" path="/var/lib/kubelet/pods/26e0b856-1a6a-45bf-9c4b-e0b1c6f2f3ae/volumes"
Oct 31 01:15:30.565134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:30.565096    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" podUID="6da29a69-9b53-4506-8db0-4758c504af67" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:15:32.155911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:32.155874    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"]
Oct 31 01:15:32.156507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:32.156476    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" containerID="cri-o://b9b02a962f0b65dbfc1715a7db65858fb78df6e308c344b1da41ac1f877f7913" gracePeriod=30
Oct 31 01:15:32.172306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:32.172268    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"]
Oct 31 01:15:32.172485 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:32.172461    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy" containerID="cri-o://65cf5c93705f670bb2ac60cf36463bce5fe6c94f0329cb009e90f9fb15c45783" gracePeriod=30
Oct 31 01:15:32.178006 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:32.177981    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"]
Oct 31 01:15:32.178160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:32.178143    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy" containerID="cri-o://a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae" gracePeriod=30
Oct 31 01:15:33.803909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:33.803876    8029 patch_prober.go:29] interesting pod/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:15:33.804370 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:33.803944    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:15:34.088828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.088798    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:15:34.245943 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.245914    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume\") pod \"6da29a69-9b53-4506-8db0-4758c504af67\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") "
Oct 31 01:15:34.246113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.245990    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn\" (UniqueName: \"kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn\") pod \"6da29a69-9b53-4506-8db0-4758c504af67\" (UID: \"6da29a69-9b53-4506-8db0-4758c504af67\") "
Oct 31 01:15:34.246343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.246319    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume" pod "6da29a69-9b53-4506-8db0-4758c504af67" (UID: "6da29a69-9b53-4506-8db0-4758c504af67"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:34.246518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.246500    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da29a69-9b53-4506-8db0-4758c504af67-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:34.248213 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.248182    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn" pod "6da29a69-9b53-4506-8db0-4758c504af67" (UID: "6da29a69-9b53-4506-8db0-4758c504af67"). InnerVolumeSpecName "kube-api-access-s2hpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:34.348127 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.348069    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2hpn\" (UniqueName: \"kubernetes.io/projected/6da29a69-9b53-4506-8db0-4758c504af67-kube-api-access-s2hpn\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:34.949382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.949344    8029 generic.go:356] "Generic (PLEG): container finished" podID="6da29a69-9b53-4506-8db0-4758c504af67" containerID="44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711" exitCode=0
Oct 31 01:15:34.949864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.949389    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"
Oct 31 01:15:34.949864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.949396    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" event={"ID":"6da29a69-9b53-4506-8db0-4758c504af67","Type":"ContainerDied","Data":"44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711"}
Oct 31 01:15:34.949864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.949440    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6" event={"ID":"6da29a69-9b53-4506-8db0-4758c504af67","Type":"ContainerDied","Data":"f788f66d4a0b76581e4bd58dba5e6640388fe2b3f436fd5ce33b0ee5363c57f0"}
Oct 31 01:15:34.949864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.949464    8029 scope.go:122] "RemoveContainer" containerID="44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711"
Oct 31 01:15:34.960425 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.960406    8029 scope.go:122] "RemoveContainer" containerID="44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711"
Oct 31 01:15:34.960737 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:34.960713    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711\": container with ID starting with 44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711 not found: ID does not exist" containerID="44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711"
Oct 31 01:15:34.960845 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.960745    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711"} err="failed to get container status \"44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711\": rpc error: code = NotFound desc = could not find container \"44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711\": container with ID starting with 44c52fd65f25efd4c4d7e24ec4a55c5f75497defd81bbcdbe74c630292d6c711 not found: ID does not exist"
Oct 31 01:15:34.973639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.973575    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"]
Oct 31 01:15:34.977413 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:34.977390    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-487a6c-d98d44547-v4fm6"]
Oct 31 01:15:35.182161 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:35.182128    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6da29a69-9b53-4506-8db0-4758c504af67" path="/var/lib/kubelet/pods/6da29a69-9b53-4506-8db0-4758c504af67/volumes"
Oct 31 01:15:37.875939 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.875560    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:15:37.976247 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976147    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.976247 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976197    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.976400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976250    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.976400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976312    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.976400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976358    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.976589 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976418    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.976589 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.976524    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.985167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.984945    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.985167 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.985007    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk\") pod \"3a7b580d-091c-4735-aedf-5e8964bbd863\" (UID: \"3a7b580d-091c-4735-aedf-5e8964bbd863\") "
Oct 31 01:15:37.997078 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:37.997046    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "kube-api-access-tzctk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:38.005656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.005412    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.026121 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.025743    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.030673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.026533    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.063199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.062535    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:38.063199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.063160    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:38.063555 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.063500    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:38.065448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.063679    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.065718 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.065676    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data" pod "3a7b580d-091c-4735-aedf-5e8964bbd863" (UID: "3a7b580d-091c-4735-aedf-5e8964bbd863"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.072384 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.072362    8029 generic.go:356] "Generic (PLEG): container finished" podID="cfae1a84-3356-4494-9064-43a79c66029f" containerID="65cf5c93705f670bb2ac60cf36463bce5fe6c94f0329cb009e90f9fb15c45783" exitCode=0
Oct 31 01:15:38.072489 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.072435    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" event={"ID":"cfae1a84-3356-4494-9064-43a79c66029f","Type":"ContainerDied","Data":"65cf5c93705f670bb2ac60cf36463bce5fe6c94f0329cb009e90f9fb15c45783"}
Oct 31 01:15:38.074377 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.074343    8029 generic.go:356] "Generic (PLEG): container finished" podID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerID="a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae" exitCode=0
Oct 31 01:15:38.074465 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.074405    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"
Oct 31 01:15:38.074531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.074399    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" event={"ID":"3a7b580d-091c-4735-aedf-5e8964bbd863","Type":"ContainerDied","Data":"a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae"}
Oct 31 01:15:38.074593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.074554    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz" event={"ID":"3a7b580d-091c-4735-aedf-5e8964bbd863","Type":"ContainerDied","Data":"c158cfbcd589b738a9d530989db922269df5730afe6adc269b5c0565be6ae081"}
Oct 31 01:15:38.074714 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.074593    8029 scope.go:122] "RemoveContainer" containerID="a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae"
Oct 31 01:15:38.075290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.075272    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:15:38.076229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.076209    8029 generic.go:356] "Generic (PLEG): container finished" podID="5eb32cdd-844b-4317-83b7-14b771223847" containerID="b9b02a962f0b65dbfc1715a7db65858fb78df6e308c344b1da41ac1f877f7913" exitCode=0
Oct 31 01:15:38.076299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.076282    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" event={"ID":"5eb32cdd-844b-4317-83b7-14b771223847","Type":"ContainerDied","Data":"b9b02a962f0b65dbfc1715a7db65858fb78df6e308c344b1da41ac1f877f7913"}
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089327    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089357    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089376    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089390    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089404    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089418    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a7b580d-091c-4735-aedf-5e8964bbd863-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089431    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089445    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a7b580d-091c-4735-aedf-5e8964bbd863-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.090639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.089461    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzctk\" (UniqueName: \"kubernetes.io/projected/3a7b580d-091c-4735-aedf-5e8964bbd863-kube-api-access-tzctk\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.148740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.148437    8029 scope.go:122] "RemoveContainer" containerID="a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae"
Oct 31 01:15:38.149323 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:38.148990    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae\": container with ID starting with a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae not found: ID does not exist" containerID="a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae"
Oct 31 01:15:38.149323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.149027    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae"} err="failed to get container status \"a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae\": rpc error: code = NotFound desc = could not find container \"a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae\": container with ID starting with a396ca7e312608ce4226e667e654fff16f2e53dc6b31d88b6b4757e027f9c9ae not found: ID does not exist"
Oct 31 01:15:38.149323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.149054    8029 scope.go:122] "RemoveContainer" containerID="b9b02a962f0b65dbfc1715a7db65858fb78df6e308c344b1da41ac1f877f7913"
Oct 31 01:15:38.152801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.152773    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194009    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194064    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194110    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194166    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194213    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194254    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194295    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194323    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194362    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194435    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194497    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194551    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194589    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194638    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194667    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194712    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194744    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data\") pod \"5eb32cdd-844b-4317-83b7-14b771223847\" (UID: \"5eb32cdd-844b-4317-83b7-14b771223847\") "
Oct 31 01:15:38.213564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.194778    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm\") pod \"cfae1a84-3356-4494-9064-43a79c66029f\" (UID: \"cfae1a84-3356-4494-9064-43a79c66029f\") "
Oct 31 01:15:38.266996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.266655    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.269327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.269285    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.269536 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.269517    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"]
Oct 31 01:15:38.269637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.269566    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-5nzbz"]
Oct 31 01:15:38.269637 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.269580    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:38.269892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.269870    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "kube-api-access-trtcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:38.270204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.270185    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.270387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.270365    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.270590 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.270571    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.271973 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.271770    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "kube-api-access-m57xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:38.272703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.272128    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.273301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.273281    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:15:38.273661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.273525    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.273786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.273756    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:38.273934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.273908    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.274507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.274475    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:15:38.275151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.275121    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.275850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.275824    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:38.275975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.275952    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo" pod "cfae1a84-3356-4494-9064-43a79c66029f" (UID: "cfae1a84-3356-4494-9064-43a79c66029f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:15:38.276075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.276059    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy" pod "5eb32cdd-844b-4317-83b7-14b771223847" (UID: "5eb32cdd-844b-4317-83b7-14b771223847"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299466    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299498    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299513    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb32cdd-844b-4317-83b7-14b771223847-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299527    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299543    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299557    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299573    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299587    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfae1a84-3356-4494-9064-43a79c66029f-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299623    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299639    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trtcm\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-kube-api-access-trtcm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299654    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299668    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57xp\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-kube-api-access-m57xp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299682    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb32cdd-844b-4317-83b7-14b771223847-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299697    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb32cdd-844b-4317-83b7-14b771223847-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299711    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb32cdd-844b-4317-83b7-14b771223847-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299726    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfae1a84-3356-4494-9064-43a79c66029f-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299740    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfae1a84-3356-4494-9064-43a79c66029f-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:38.337581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:38.299753    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfae1a84-3356-4494-9064-43a79c66029f-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:15:39.085767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.085736    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"
Oct 31 01:15:39.085767 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.085742    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4" event={"ID":"cfae1a84-3356-4494-9064-43a79c66029f","Type":"ContainerDied","Data":"d1501bbc4440175f65a9e8b84e9cbf89541ce3e55d826e6c1c35c37da6f75b71"}
Oct 31 01:15:39.086318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.085816    8029 scope.go:122] "RemoveContainer" containerID="65cf5c93705f670bb2ac60cf36463bce5fe6c94f0329cb009e90f9fb15c45783"
Oct 31 01:15:39.088118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.088095    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"
Oct 31 01:15:39.088262 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.088126    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w" event={"ID":"5eb32cdd-844b-4317-83b7-14b771223847","Type":"ContainerDied","Data":"35a5da4e8a625ddd4645b3340c0f1559a1513c64199c5c7900e612daafeebf0e"}
Oct 31 01:15:39.110950 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.110917    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"]
Oct 31 01:15:39.114864 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.114843    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-c6gg4"]
Oct 31 01:15:39.125790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.125754    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"]
Oct 31 01:15:39.129073 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.129051    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-487a6c-openshift-default-567f7cfc97-xcs5w"]
Oct 31 01:15:39.181996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.181968    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" path="/var/lib/kubelet/pods/3a7b580d-091c-4735-aedf-5e8964bbd863/volumes"
Oct 31 01:15:39.182401 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.182388    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5eb32cdd-844b-4317-83b7-14b771223847" path="/var/lib/kubelet/pods/5eb32cdd-844b-4317-83b7-14b771223847/volumes"
Oct 31 01:15:39.182783 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:39.182771    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfae1a84-3356-4494-9064-43a79c66029f" path="/var/lib/kubelet/pods/cfae1a84-3356-4494-9064-43a79c66029f/volumes"
Oct 31 01:15:44.784280 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:15:44.784249    8029 scope.go:122] "RemoveContainer" containerID="93a008dfe697355201a758ba5f7c886c3e6af1f50922473c02080672a73937a6"
Oct 31 01:15:49.441798 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:15:49.441754    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:16:06.029259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:06.028895    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"]
Oct 31 01:16:06.033681 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:06.033647    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"]
Oct 31 01:16:06.036705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:06.036689    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d951865f-4ftjf"]
Oct 31 01:16:06.039640 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:06.039611    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eed1f1d4-gzljw"]
Oct 31 01:16:07.182258 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:07.182229    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2061e09-f874-45ad-9873-c5eb5e6ee96f" path="/var/lib/kubelet/pods/f2061e09-f874-45ad-9873-c5eb5e6ee96f/volumes"
Oct 31 01:16:07.182760 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:07.182745    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd0bca43-56d1-43fb-8abb-f1442f4abcbe" path="/var/lib/kubelet/pods/fd0bca43-56d1-43fb-8abb-f1442f4abcbe/volumes"
Oct 31 01:16:12.028772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:12.028714    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"]
Oct 31 01:16:12.029272 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:12.029043    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e8b2b56-6jkdm"]
Oct 31 01:16:13.020207 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:13.020173    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"]
Oct 31 01:16:13.023657 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:13.023621    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4e58b98-6crzn"]
Oct 31 01:16:13.182578 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:13.182547    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5186862a-65f0-4650-82bc-999bbe3c6351" path="/var/lib/kubelet/pods/5186862a-65f0-4650-82bc-999bbe3c6351/volumes"
Oct 31 01:16:13.182995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:13.182981    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aafdd6c2-786d-48d7-98c3-4dbe96d40a99" path="/var/lib/kubelet/pods/aafdd6c2-786d-48d7-98c3-4dbe96d40a99/volumes"
Oct 31 01:16:15.031746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:15.031692    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"]
Oct 31 01:16:15.035917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:15.035895    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9f4eec7a-mf5xq"]
Oct 31 01:16:15.181510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:15.181478    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaff83dd-8191-43d6-af5c-1a4f93116525" path="/var/lib/kubelet/pods/aaff83dd-8191-43d6-af5c-1a4f93116525/volumes"
Oct 31 01:16:16.023206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:16.023172    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"]
Oct 31 01:16:16.026474 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:16.026451    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7075181b-g7sqh"]
Oct 31 01:16:17.182032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:17.182003    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df0e1084-caf2-4e4c-ad78-70519dc3d864" path="/var/lib/kubelet/pods/df0e1084-caf2-4e4c-ad78-70519dc3d864/volumes"
Oct 31 01:16:19.021331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:19.021249    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"]
Oct 31 01:16:19.024524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:19.024501    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4e43c771-22zh9"]
Oct 31 01:16:19.182311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:19.182274    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bcb1430-31ce-4de6-929b-237879a0a46b" path="/var/lib/kubelet/pods/1bcb1430-31ce-4de6-929b-237879a0a46b/volumes"
Oct 31 01:16:19.442286 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:16:19.442252    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:16:23.021824 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:23.021788    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"]
Oct 31 01:16:23.025091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:23.025068    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-feb4ac6b-sfnww"]
Oct 31 01:16:23.183381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:23.183347    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="604a805b-8913-4f7b-a750-b9e1a6d2a36c" path="/var/lib/kubelet/pods/604a805b-8913-4f7b-a750-b9e1a6d2a36c/volumes"
Oct 31 01:16:24.022448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:24.022403    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"]
Oct 31 01:16:24.025799 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:24.025766    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f347c451-885rr"]
Oct 31 01:16:25.183044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:25.183011    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="879be096-2fc1-4724-9702-feaa32fb8323" path="/var/lib/kubelet/pods/879be096-2fc1-4724-9702-feaa32fb8323/volumes"
Oct 31 01:16:26.020178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:26.020145    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"]
Oct 31 01:16:26.023229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:26.023211    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-626021dd-jvl4g"]
Oct 31 01:16:27.182481 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:27.182444    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d959f49-7094-44a3-84ca-1f236606c356" path="/var/lib/kubelet/pods/7d959f49-7094-44a3-84ca-1f236606c356/volumes"
Oct 31 01:16:28.021252 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:28.021210    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"]
Oct 31 01:16:28.024293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:28.024268    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8a73b0b2-g2cn4"]
Oct 31 01:16:29.022919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:29.022884    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"]
Oct 31 01:16:29.026012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:29.025991    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ed8f450-h82v7"]
Oct 31 01:16:29.182056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:29.182020    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a53a862-9a21-4196-9011-270d98389276" path="/var/lib/kubelet/pods/2a53a862-9a21-4196-9011-270d98389276/volumes"
Oct 31 01:16:29.182491 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:29.182470    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="992b3425-4206-49cf-8342-213c251d0504" path="/var/lib/kubelet/pods/992b3425-4206-49cf-8342-213c251d0504/volumes"
Oct 31 01:16:30.021862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:30.021831    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"]
Oct 31 01:16:30.024848 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:30.024825    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c70795f8-kpx4s"]
Oct 31 01:16:31.182042 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:31.182009    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5a18cee-2322-4250-8f46-84225409131b" path="/var/lib/kubelet/pods/f5a18cee-2322-4250-8f46-84225409131b/volumes"
Oct 31 01:16:38.023705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:38.023665    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"]
Oct 31 01:16:38.027082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:38.027056    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"]
Oct 31 01:16:38.029680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:38.029663    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75d2b1a8-72jqt"]
Oct 31 01:16:38.032134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:38.032110    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5cb495a-p8nhk"]
Oct 31 01:16:39.023520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.023478    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"]
Oct 31 01:16:39.028024 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.027986    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"]
Oct 31 01:16:39.030628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.030607    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0dcc5e16-6vjhk"]
Oct 31 01:16:39.033085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.033064    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f91ed2b0-2zs8r"]
Oct 31 01:16:39.181775 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.181734    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26d94cd1-e495-488a-89ae-9a58c4d73762" path="/var/lib/kubelet/pods/26d94cd1-e495-488a-89ae-9a58c4d73762/volumes"
Oct 31 01:16:39.182229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.182208    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c72b57f-dc1d-477f-8db7-3866843652ae" path="/var/lib/kubelet/pods/2c72b57f-dc1d-477f-8db7-3866843652ae/volumes"
Oct 31 01:16:39.182666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.182648    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0049f5b-acd7-47d7-a4f8-6e3967b89708" path="/var/lib/kubelet/pods/a0049f5b-acd7-47d7-a4f8-6e3967b89708/volumes"
Oct 31 01:16:39.183381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:39.183360    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae7be827-646d-439a-92cf-dafb2e988e0b" path="/var/lib/kubelet/pods/ae7be827-646d-439a-92cf-dafb2e988e0b/volumes"
Oct 31 01:16:41.021044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:41.021002    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"]
Oct 31 01:16:41.025324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:41.025298    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d47f5ee-kzv4h"]
Oct 31 01:16:41.181842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:41.181808    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="828145cb-f190-4925-b4a9-c723ea6dcb60" path="/var/lib/kubelet/pods/828145cb-f190-4925-b4a9-c723ea6dcb60/volumes"
Oct 31 01:16:43.403977 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:43.403949    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:16:43.406968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:43.406944    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:16:49.446311 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:16:49.446265    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:16:54.022043 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:54.021993    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"]
Oct 31 01:16:54.025002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:54.024974    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7487971e-dvvbq"]
Oct 31 01:16:55.181728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:16:55.181694    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5" path="/var/lib/kubelet/pods/0218cbb3-128f-47fe-bcc9-a6b1f29e7ec5/volumes"
Oct 31 01:17:04.614903 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:04.614867    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:17:04.615503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:04.615470    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:17:05.042389 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:05.042297    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"]
Oct 31 01:17:05.052475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:05.052427    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4cf2b4d9-jh2db"]
Oct 31 01:17:05.181818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:05.181790    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="490b1bf5-e143-4bde-9696-35b49dd2b637" path="/var/lib/kubelet/pods/490b1bf5-e143-4bde-9696-35b49dd2b637/volumes"
Oct 31 01:17:19.443797 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:17:19.443750    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:17:27.021620 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:27.021567    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"]
Oct 31 01:17:27.024433 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:27.024410    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5eb8a263-9hk29"]
Oct 31 01:17:27.182007 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:27.181974    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03f30b7e-ac75-4800-88c2-788a51708025" path="/var/lib/kubelet/pods/03f30b7e-ac75-4800-88c2-788a51708025/volumes"
Oct 31 01:17:34.617311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:34.617247    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:17:34.617998 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:34.617965    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:17:40.021118 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:40.021077    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"]
Oct 31 01:17:40.024108 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:40.024080    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-206b91f3-n2m67"]
Oct 31 01:17:41.182267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:41.182235    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86cba2b7-af0f-4a27-889f-cd6b82bc63ad" path="/var/lib/kubelet/pods/86cba2b7-af0f-4a27-889f-cd6b82bc63ad/volumes"
Oct 31 01:17:44.958854 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:44.958826    8029 scope.go:122] "RemoveContainer" containerID="43b365ae883651edebd8a99bdb7f0c5d8162afad2b5869060025c6b4597870cf"
Oct 31 01:17:44.969367 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:44.969336    8029 scope.go:122] "RemoveContainer" containerID="e35004411a0507b3feff3b8e14b8161f95f5efc7e42f5b2a26e1fea7f2dc81ca"
Oct 31 01:17:44.978921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:44.978903    8029 scope.go:122] "RemoveContainer" containerID="b9b466e436f4e374ecea292f4ac7e96a5ccd32645bdf1dabbd8e4fe36e73d20e"
Oct 31 01:17:44.987829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:44.987806    8029 scope.go:122] "RemoveContainer" containerID="8148d56b7e85285b1d87fb89809223f946c797cfdbc934c81a0a65ce40442132"
Oct 31 01:17:44.999660 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:44.999632    8029 scope.go:122] "RemoveContainer" containerID="1daaa26379668dbb1410061a71fa9110317175686b7694c69f7b2291bc63f64b"
Oct 31 01:17:45.009430 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.009410    8029 scope.go:122] "RemoveContainer" containerID="6fcd384f2c07894ce082468fdc3524dac8d62dc056d9a47065aca5b2770572b0"
Oct 31 01:17:45.018440 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.018417    8029 scope.go:122] "RemoveContainer" containerID="daeec6560bd5a25fa109e59def75fa07f7da3f9dbefc356957465cbc4996b511"
Oct 31 01:17:45.027657 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.027635    8029 scope.go:122] "RemoveContainer" containerID="ab9642f0fc5e60f89b57d6b2810aa2e86b589a4305f6aed12b9a90595360007c"
Oct 31 01:17:45.036382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.036362    8029 scope.go:122] "RemoveContainer" containerID="3dffaa0968a6823ee97b8112a9ed74f38b8dc646f576b9d2f8be74728b054b6b"
Oct 31 01:17:45.045852 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.045832    8029 scope.go:122] "RemoveContainer" containerID="75b2f21d4f0f2de15d4bbdfc7fc286d154ed00ed42dab250692586d0bb6d505f"
Oct 31 01:17:45.054351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.054334    8029 scope.go:122] "RemoveContainer" containerID="38a32b0b1501ec375ace541c6f22a5fe68ad83a637e88075d76c789c33985174"
Oct 31 01:17:45.063016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.062990    8029 scope.go:122] "RemoveContainer" containerID="c2923a4af0a96b47ba8bd69f1182c6b962b8a8289ff0d6755329b9ab86eae232"
Oct 31 01:17:45.072530 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.072511    8029 scope.go:122] "RemoveContainer" containerID="c3311add1d9319c88eff8a5435ccefe21b21f7edaf64688c06e3139121550f0a"
Oct 31 01:17:45.081590 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.081572    8029 scope.go:122] "RemoveContainer" containerID="17d934f9c41dbe8aefc549ce4cf3b7638d47043ee4458062c25b5c322b677224"
Oct 31 01:17:45.090483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.090465    8029 scope.go:122] "RemoveContainer" containerID="c5aab19dc400757d340f03643a1286f742a6af8e4226a542d0b4aedd5e36d576"
Oct 31 01:17:45.099729 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.099713    8029 scope.go:122] "RemoveContainer" containerID="5f68ee52d1da5db9879090e9d409bfd439104c70b8a49308ac4e776e15c06bff"
Oct 31 01:17:45.108451 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.108434    8029 scope.go:122] "RemoveContainer" containerID="24fbbb026f4caa1940012788ec27c95d4015e59f3f8f0dab501e5daca8b7ca0c"
Oct 31 01:17:45.117174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:45.117155    8029 scope.go:122] "RemoveContainer" containerID="2123a824ff6510b9c11ee4c3fe5eca891fe4df8d7949d8d26f2c21ba12858f44"
Oct 31 01:17:49.442083 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:17:49.441998    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:17:57.023318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:57.023252    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"]
Oct 31 01:17:57.027979 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:57.027944    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-913e10f0-jb7mb"]
Oct 31 01:17:57.184189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:17:57.184152    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de8f2059-f99c-47bf-8a1c-132b2dc4c63c" path="/var/lib/kubelet/pods/de8f2059-f99c-47bf-8a1c-132b2dc4c63c/volumes"
Oct 31 01:18:00.017233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:00.017200    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"]
Oct 31 01:18:00.020152 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:00.020126    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e52cb5f6-t5j4w"]
Oct 31 01:18:01.182784 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:01.182750    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e84812af-980f-430f-b321-31787969eb84" path="/var/lib/kubelet/pods/e84812af-980f-430f-b321-31787969eb84/volumes"
Oct 31 01:18:05.018195 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:05.018159    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"]
Oct 31 01:18:05.021055 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:05.021030    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76d9dc0b-xqqd5"]
Oct 31 01:18:05.181779 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:05.181747    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12" path="/var/lib/kubelet/pods/56bbb45f-ddbc-44c4-a5b7-e9e4d7e2fa12/volumes"
Oct 31 01:18:14.017016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:14.016979    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"]
Oct 31 01:18:14.019435 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:14.019415    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d34d949-8z68p"]
Oct 31 01:18:15.028102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:15.028070    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"]
Oct 31 01:18:15.030335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:15.030312    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-183fb75c-mmdd2"]
Oct 31 01:18:15.181975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:15.181943    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5752fcac-a881-4c0e-baac-37d39a2fce9e" path="/var/lib/kubelet/pods/5752fcac-a881-4c0e-baac-37d39a2fce9e/volumes"
Oct 31 01:18:15.182355 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:15.182342    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f05fafde-53c9-4a5e-91f3-5659c551add2" path="/var/lib/kubelet/pods/f05fafde-53c9-4a5e-91f3-5659c551add2/volumes"
Oct 31 01:18:16.024624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:16.024111    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"]
Oct 31 01:18:16.032925 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:16.032893    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f82041d-6b7sg"]
Oct 31 01:18:17.183436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:17.183405    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d87eef85-111c-4ed6-afd9-3245cb0929e5" path="/var/lib/kubelet/pods/d87eef85-111c-4ed6-afd9-3245cb0929e5/volumes"
Oct 31 01:18:19.442313 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:18:19.442278    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:18:22.022855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:22.022814    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"]
Oct 31 01:18:22.026186 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:22.026156    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd0ee2b3-hktqf"]
Oct 31 01:18:23.182740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:23.182710    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1edab968-fd36-450c-bc2c-1f07648b642d" path="/var/lib/kubelet/pods/1edab968-fd36-450c-bc2c-1f07648b642d/volumes"
Oct 31 01:18:45.229056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:45.229026    8029 scope.go:122] "RemoveContainer" containerID="90fee6d375f87f0f557e0be384f5560bb769c5b1501e79f52b8df14ae3d5616a"
Oct 31 01:18:45.240627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:45.240562    8029 scope.go:122] "RemoveContainer" containerID="d628710a306152df69507ebd9a19581db8c9d30791784a86f8fa7f4311f4dba5"
Oct 31 01:18:45.249776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:45.249758    8029 scope.go:122] "RemoveContainer" containerID="9b43d07f05407a34cc93546aa56e8efbba2c578281e5f5a0dbc0f53f2b5eb886"
Oct 31 01:18:45.258002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:45.257987    8029 scope.go:122] "RemoveContainer" containerID="a3db475550412fbb2d397c258ea6876682690cca52879b115948b121bdae4a99"
Oct 31 01:18:49.470458 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:18:49.470418    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:18:53.028149 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:53.028102    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"]
Oct 31 01:18:53.028663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:53.028171    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9d0fa3f3-kcxjf"]
Oct 31 01:18:53.181663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:53.181627    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9d055e5-9b62-404b-8c99-46f4c0ac91d9" path="/var/lib/kubelet/pods/d9d055e5-9b62-404b-8c99-46f4c0ac91d9/volumes"
Oct 31 01:18:54.021439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:54.021401    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"]
Oct 31 01:18:54.027271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:54.027243    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c46d0f47-hsxhw"]
Oct 31 01:18:54.029471 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:54.029451    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"]
Oct 31 01:18:54.031331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:54.031311    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c26e61b-2h2tj"]
Oct 31 01:18:55.181703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:55.181674    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38f56b97-0180-4da3-b24b-6b4cc112e60c" path="/var/lib/kubelet/pods/38f56b97-0180-4da3-b24b-6b4cc112e60c/volumes"
Oct 31 01:18:55.182079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:55.182066    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac0ca893-722d-4c70-8616-86035ddfbc33" path="/var/lib/kubelet/pods/ac0ca893-722d-4c70-8616-86035ddfbc33/volumes"
Oct 31 01:18:57.020159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:57.020124    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"]
Oct 31 01:18:57.023093 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:57.023066    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6d54aef9-97tvp"]
Oct 31 01:18:57.182333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:57.182305    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e88ecd35-15ca-4dab-89e3-d3d02f4264a9" path="/var/lib/kubelet/pods/e88ecd35-15ca-4dab-89e3-d3d02f4264a9/volumes"
Oct 31 01:18:58.025885 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:58.025848    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"]
Oct 31 01:18:58.028800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:58.028776    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a5d2acf-z87qc"]
Oct 31 01:18:59.182170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:18:59.182138    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f47907ba-7a0c-4fd1-860c-7d1bccde84be" path="/var/lib/kubelet/pods/f47907ba-7a0c-4fd1-860c-7d1bccde84be/volumes"
Oct 31 01:19:01.018818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:01.018769    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"]
Oct 31 01:19:01.021374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:01.021349    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbdb7259-6tlt6"]
Oct 31 01:19:01.181405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:01.181373    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abb2c902-5b24-4658-afdd-a9e853d5857e" path="/var/lib/kubelet/pods/abb2c902-5b24-4658-afdd-a9e853d5857e/volumes"
Oct 31 01:19:02.019849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:02.019816    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"]
Oct 31 01:19:02.022030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:02.022009    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c61cb88f-j4kmc"]
Oct 31 01:19:03.182278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:03.182245    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b70060f-9a58-491d-a2cb-83349d787957" path="/var/lib/kubelet/pods/0b70060f-9a58-491d-a2cb-83349d787957/volumes"
Oct 31 01:19:10.023291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:10.023253    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"]
Oct 31 01:19:10.026278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:10.026252    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"]
Oct 31 01:19:10.028148 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:10.028134    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-627c06ba-xhkkc"]
Oct 31 01:19:10.029740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:10.029722    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7bad7863-kjpv8"]
Oct 31 01:19:11.181994 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:11.181964    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a74cbf15-4be0-4b33-9e40-d36950c640ec" path="/var/lib/kubelet/pods/a74cbf15-4be0-4b33-9e40-d36950c640ec/volumes"
Oct 31 01:19:11.182374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:11.182332    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df8c649f-ed6e-4d5c-9f83-959d51b79706" path="/var/lib/kubelet/pods/df8c649f-ed6e-4d5c-9f83-959d51b79706/volumes"
Oct 31 01:19:15.033217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:15.033179    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"]
Oct 31 01:19:15.035102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:15.035080    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ee0c41c-68rgc"]
Oct 31 01:19:15.182614 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:15.182573    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e1dc8f5-0b89-4577-968b-c846879dc0ff" path="/var/lib/kubelet/pods/8e1dc8f5-0b89-4577-968b-c846879dc0ff/volumes"
Oct 31 01:19:16.037586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:16.037551    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"]
Oct 31 01:19:16.044682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:16.044659    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f834682-ndbwc"]
Oct 31 01:19:17.181370 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:17.181337    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25f965ef-9029-46ca-af46-7503825eaafb" path="/var/lib/kubelet/pods/25f965ef-9029-46ca-af46-7503825eaafb/volumes"
Oct 31 01:19:19.441553 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:19:19.441472    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:19:22.021031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:22.021000    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"]
Oct 31 01:19:22.025411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:22.025382    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-12a087ad-zlmb6"]
Oct 31 01:19:23.182432 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:23.182402    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b67dbb9-2291-4645-9b71-926b6b936296" path="/var/lib/kubelet/pods/0b67dbb9-2291-4645-9b71-926b6b936296/volumes"
Oct 31 01:19:24.021908 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:24.021869    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"]
Oct 31 01:19:24.023617 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:24.023577    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1b23d49-9bflt"]
Oct 31 01:19:25.182320 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:25.182286    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="792d3fb7-b6f8-47fc-9017-e2a300023f16" path="/var/lib/kubelet/pods/792d3fb7-b6f8-47fc-9017-e2a300023f16/volumes"
Oct 31 01:19:29.022865 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:29.022834    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"]
Oct 31 01:19:29.029543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:29.029514    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73974aa0-2562q"]
Oct 31 01:19:29.029736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:29.029562    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"]
Oct 31 01:19:29.031194 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:29.031172    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f95bdf2-d7lnk"]
Oct 31 01:19:29.181516 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:29.181489    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="063a2658-5310-4b07-bbe6-b87b7d875940" path="/var/lib/kubelet/pods/063a2658-5310-4b07-bbe6-b87b7d875940/volumes"
Oct 31 01:19:29.181905 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:29.181891    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="510bd72f-cad7-4cb0-a550-5fc9031ef218" path="/var/lib/kubelet/pods/510bd72f-cad7-4cb0-a550-5fc9031ef218/volumes"
Oct 31 01:19:36.062882 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:36.062845    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"]
Oct 31 01:19:36.066547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:36.066521    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"]
Oct 31 01:19:36.080935 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:36.080900    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cec76339-j97kz"]
Oct 31 01:19:36.082279 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:36.082253    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78e20883-26dxz"]
Oct 31 01:19:36.090395 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:36.090375    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"]
Oct 31 01:19:36.098929 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:36.098906    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-10e06873-q9628"]
Oct 31 01:19:37.186674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:37.186632    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c601844-fa2c-4865-befd-03f0dc788725" path="/var/lib/kubelet/pods/3c601844-fa2c-4865-befd-03f0dc788725/volumes"
Oct 31 01:19:37.187227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:37.187214    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3feb32fa-9d2d-4d52-b778-40e57217bbfa" path="/var/lib/kubelet/pods/3feb32fa-9d2d-4d52-b778-40e57217bbfa/volumes"
Oct 31 01:19:37.187593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:37.187579    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="743abb0c-6a7d-428d-9fae-c153f886345b" path="/var/lib/kubelet/pods/743abb0c-6a7d-428d-9fae-c153f886345b/volumes"
Oct 31 01:19:45.304292 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.304260    8029 scope.go:122] "RemoveContainer" containerID="505e172248f77bd64b53ec086b2d44612b4c086e2c673567bf65c952f953393c"
Oct 31 01:19:45.313940 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.313922    8029 scope.go:122] "RemoveContainer" containerID="c57e64fec6639ffba49a8808879135ae95cf2bdabd3e446b3c605a9f422e9ec1"
Oct 31 01:19:45.324872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.324849    8029 scope.go:122] "RemoveContainer" containerID="402cecd08a5f5dc2a1f755463a9f2695da1bd88fb57382cd5580527f3575c24c"
Oct 31 01:19:45.332996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.332980    8029 scope.go:122] "RemoveContainer" containerID="767f37acc4134f0758a740ea3ec64a4d2dc223f8c4984af5525b8045882fdf06"
Oct 31 01:19:45.341416 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.341398    8029 scope.go:122] "RemoveContainer" containerID="ef65f0b77ef15fd0819b35edba37d4f2d2016b8fa3f9a3f44a73a9992f73de17"
Oct 31 01:19:45.348921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.348906    8029 scope.go:122] "RemoveContainer" containerID="d781b70a1d08272680620b926b24d0ad2d437dd772605b583ce70cb750e08215"
Oct 31 01:19:45.356892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.356875    8029 scope.go:122] "RemoveContainer" containerID="12de93ad7ddbcd4ebd2f652bd601906a11869864f957a34b19daa252ff8b45e2"
Oct 31 01:19:45.364408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:45.364390    8029 scope.go:122] "RemoveContainer" containerID="5b3b63e19c287e34d297a1ce82ef04136c26452234e27e1c92e0cc866bc8c1d6"
Oct 31 01:19:49.441935 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:19:49.441891    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:19:51.019966 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:51.019934    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"]
Oct 31 01:19:51.023648 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:51.023625    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"]
Oct 31 01:19:51.027172 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:51.027148    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04cde8ab-76h2c"]
Oct 31 01:19:51.029197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:51.029173    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d46506d-fcf7v"]
Oct 31 01:19:51.182157 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:51.182129    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88a225ed-f649-47bf-8496-2ad4544dfd43" path="/var/lib/kubelet/pods/88a225ed-f649-47bf-8496-2ad4544dfd43/volumes"
Oct 31 01:19:51.182508 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:19:51.182496    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cec2b33b-3e06-41a9-9c6d-a1a4e1604368" path="/var/lib/kubelet/pods/cec2b33b-3e06-41a9-9c6d-a1a4e1604368/volumes"
Oct 31 01:20:13.040124 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:13.040081    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"]
Oct 31 01:20:13.046421 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:13.046390    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"]
Oct 31 01:20:13.047577 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:13.047559    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74995c3b-wr4fn"]
Oct 31 01:20:13.050965 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:13.050943    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-771a8d91-kz5c5"]
Oct 31 01:20:13.181966 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:13.181933    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63bd58bf-c212-474b-b637-10cc13e086fe" path="/var/lib/kubelet/pods/63bd58bf-c212-474b-b637-10cc13e086fe/volumes"
Oct 31 01:20:13.182338 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:13.182326    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dab6a92-5447-4100-a239-e35458f07490" path="/var/lib/kubelet/pods/8dab6a92-5447-4100-a239-e35458f07490/volumes"
Oct 31 01:20:19.442126 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:20:19.442085    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:20:33.031271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.031238    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"]
Oct 31 01:20:33.031912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.031442    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms" podUID="fa39e042-7d7a-44df-b93a-4a12c30c8708" containerName="discovery" containerID="cri-o://8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5" gracePeriod=30
Oct 31 01:20:33.287064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.287000    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:20:33.339783 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.339743    8029 generic.go:356] "Generic (PLEG): container finished" podID="fa39e042-7d7a-44df-b93a-4a12c30c8708" containerID="8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5" exitCode=0
Oct 31 01:20:33.339955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.339790    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"
Oct 31 01:20:33.339955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.339819    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms" event={"ID":"fa39e042-7d7a-44df-b93a-4a12c30c8708","Type":"ContainerDied","Data":"8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5"}
Oct 31 01:20:33.339955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.339882    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms" event={"ID":"fa39e042-7d7a-44df-b93a-4a12c30c8708","Type":"ContainerDied","Data":"0d224ce94694a6b34fa6ad246ef17e7d1986e44db5f2cfc75666430aa88868e7"}
Oct 31 01:20:33.339955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.339909    8029 scope.go:122] "RemoveContainer" containerID="8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5"
Oct 31 01:20:33.350240 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.350223    8029 scope.go:122] "RemoveContainer" containerID="8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5"
Oct 31 01:20:33.350596 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:20:33.350574    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5\": container with ID starting with 8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5 not found: ID does not exist" containerID="8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5"
Oct 31 01:20:33.350740 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.350717    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5"} err="failed to get container status \"8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5\": rpc error: code = NotFound desc = could not find container \"8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5\": container with ID starting with 8103dc7943498139ca846424d4a5ff8dde206c4cf326801a8e503797dc425fe5 not found: ID does not exist"
Oct 31 01:20:33.371968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.371940    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.371982    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372110 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372052    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372112    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372185    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372211    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372340 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372288    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert\") pod \"fa39e042-7d7a-44df-b93a-4a12c30c8708\" (UID: \"fa39e042-7d7a-44df-b93a-4a12c30c8708\") "
Oct 31 01:20:33.372401 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372364    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:20:33.372702 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.372677    8029 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-ca-configmap\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.374581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.374556    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:20:33.374581 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.374564    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:20:33.374986 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.374963    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:20:33.375037 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.374994    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:20:33.375075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.375064    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "kube-api-access-49hvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:33.375107 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.375091    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token" pod "fa39e042-7d7a-44df-b93a-4a12c30c8708" (UID: "fa39e042-7d7a-44df-b93a-4a12c30c8708"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:33.473551 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.473513    8029 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-kubeconfig\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.473551 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.473543    8029 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa39e042-7d7a-44df-b93a-4a12c30c8708-local-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.473551 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.473556    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49hvn\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-kube-api-access-49hvn\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.473842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.473571    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.473842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.473584    8029 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-istio-csr-dns-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.473842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.473595    8029 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa39e042-7d7a-44df-b93a-4a12c30c8708-cacerts\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:33.665134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.665098    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"]
Oct 31 01:20:33.670805 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:33.670779    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7kbms"]
Oct 31 01:20:34.896922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.896883    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"]
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897278    8029 cpu_manager.go:414] "containerMap: removing container" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897288    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897303    8029 cpu_manager.go:414] "containerMap: removing container" podUID="8dab6a92-5447-4100-a239-e35458f07490" containerName="revoke-keys"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897308    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dab6a92-5447-4100-a239-e35458f07490" containerName="revoke-keys"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897318    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897323    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897333    8029 cpu_manager.go:414] "containerMap: removing container" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897337    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897343    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fa39e042-7d7a-44df-b93a-4a12c30c8708" containerName="discovery"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897347    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa39e042-7d7a-44df-b93a-4a12c30c8708" containerName="discovery"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897355    8029 cpu_manager.go:414] "containerMap: removing container" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897360    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897366    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerName="payload-processing"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897370    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerName="payload-processing"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897376    8029 cpu_manager.go:414] "containerMap: removing container" podUID="63bd58bf-c212-474b-b637-10cc13e086fe" containerName="revoke-keys"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897380    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63bd58bf-c212-474b-b637-10cc13e086fe" containerName="revoke-keys"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897391    8029 cpu_manager.go:414] "containerMap: removing container" podUID="6da29a69-9b53-4506-8db0-4758c504af67" containerName="payload-pre-processing"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897395    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6da29a69-9b53-4506-8db0-4758c504af67" containerName="payload-pre-processing"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897405    8029 cpu_manager.go:414] "containerMap: removing container" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897409    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897423    8029 cpu_manager.go:414] "containerMap: removing container" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897428    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy"
Oct 31 01:20:34.897422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897440    8029 cpu_manager.go:414] "containerMap: removing container" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897445    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897452    8029 cpu_manager.go:414] "containerMap: removing container" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897457    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897513    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6da29a69-9b53-4506-8db0-4758c504af67" containerName="payload-pre-processing"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897522    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="87c94909-be59-4713-8244-3444fe3c8b0a" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897531    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfae1a84-3356-4494-9064-43a79c66029f" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897537    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3288e4f3-0914-457e-bb66-ba36c4c2edc8" containerName="main"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897542    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eb32cdd-844b-4317-83b7-14b771223847" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897548    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a7b580d-091c-4735-aedf-5e8964bbd863" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897554    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dab6a92-5447-4100-a239-e35458f07490" containerName="revoke-keys"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897560    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9658b33-0dd9-42b5-9e2f-4e406b19b988" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897565    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="67747552-c23c-449f-b7dd-968dbd0e1020" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897569    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa39e042-7d7a-44df-b93a-4a12c30c8708" containerName="discovery"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897574    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="05bb1ced-152a-488d-8b65-d70892db4c2e" containerName="istio-proxy"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897579    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ef9a1a2-af7b-4e49-b6b2-6bd0588b4dc9" containerName="payload-processing"
Oct 31 01:20:34.898672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.897584    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="63bd58bf-c212-474b-b637-10cc13e086fe" containerName="revoke-keys"
Oct 31 01:20:34.900917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.900895    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:34.908531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.908501    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:20:34.908739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.908579    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:20:34.908739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.908586    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:20:34.908872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.908790    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:20:34.911325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.911291    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"]
Oct 31 01:20:34.990645 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.990561    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnmlr\" (UniqueName: \"kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:34.990842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.990698    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:34.990842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:34.990764    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.091945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.091906    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnmlr\" (UniqueName: \"kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.092188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.091997    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.092188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.092114    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.092812 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.092786    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.094544 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.094521    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.106017 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.105980    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnmlr\" (UniqueName: \"kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr\") pod \"maas-api-revoke-sub-3d4156f1-4xxlf\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.181971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.181891    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa39e042-7d7a-44df-b93a-4a12c30c8708" path="/var/lib/kubelet/pods/fa39e042-7d7a-44df-b93a-4a12c30c8708/volumes"
Oct 31 01:20:35.216919 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.216872    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:35.501569 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:35.501490    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"]
Oct 31 01:20:35.505648 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:20:35.505614    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bcc3c2_b758_42f2_9625_3f6c94162cb2.slice/crio-6b6e8d7f69d00f40466dd9ac7979285d70536a1ba64e3fa5ca1619325faa4ca1 WatchSource:0}: Error finding container 6b6e8d7f69d00f40466dd9ac7979285d70536a1ba64e3fa5ca1619325faa4ca1: Status 404 returned error can't find the container with id 6b6e8d7f69d00f40466dd9ac7979285d70536a1ba64e3fa5ca1619325faa4ca1
Oct 31 01:20:36.360406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:36.360378    8029 generic.go:356] "Generic (PLEG): container finished" podID="73bcc3c2-b758-42f2-9625-3f6c94162cb2" containerID="375e979ac4a7b6c4e67b5f8666b937133765bb8b1e8a00383a28db6b32695be6" exitCode=0
Oct 31 01:20:36.360858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:36.360457    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf" event={"ID":"73bcc3c2-b758-42f2-9625-3f6c94162cb2","Type":"ContainerDied","Data":"375e979ac4a7b6c4e67b5f8666b937133765bb8b1e8a00383a28db6b32695be6"}
Oct 31 01:20:36.360858 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:36.360521    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf" event={"ID":"73bcc3c2-b758-42f2-9625-3f6c94162cb2","Type":"ContainerStarted","Data":"6b6e8d7f69d00f40466dd9ac7979285d70536a1ba64e3fa5ca1619325faa4ca1"}
Oct 31 01:20:37.500388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.500358    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:37.618708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.618597    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca\") pod \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") "
Oct 31 01:20:37.618708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.618674    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr\" (UniqueName: \"kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr\") pod \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") "
Oct 31 01:20:37.618946 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.618735    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca\") pod \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\" (UID: \"73bcc3c2-b758-42f2-9625-3f6c94162cb2\") "
Oct 31 01:20:37.619131 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.619108    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca" pod "73bcc3c2-b758-42f2-9625-3f6c94162cb2" (UID: "73bcc3c2-b758-42f2-9625-3f6c94162cb2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:20:37.620914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.620890    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca" pod "73bcc3c2-b758-42f2-9625-3f6c94162cb2" (UID: "73bcc3c2-b758-42f2-9625-3f6c94162cb2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:20:37.620914 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.620908    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr" pod "73bcc3c2-b758-42f2-9625-3f6c94162cb2" (UID: "73bcc3c2-b758-42f2-9625-3f6c94162cb2"). InnerVolumeSpecName "kube-api-access-fnmlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:20:37.720490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.720452    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:37.720490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.720485    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnmlr\" (UniqueName: \"kubernetes.io/projected/73bcc3c2-b758-42f2-9625-3f6c94162cb2-kube-api-access-fnmlr\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:37.720490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:37.720495    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/73bcc3c2-b758-42f2-9625-3f6c94162cb2-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:20:38.372735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:38.372708    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"
Oct 31 01:20:38.372735 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:38.372724    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf" event={"ID":"73bcc3c2-b758-42f2-9625-3f6c94162cb2","Type":"ContainerDied","Data":"6b6e8d7f69d00f40466dd9ac7979285d70536a1ba64e3fa5ca1619325faa4ca1"}
Oct 31 01:20:38.372953 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:38.372768    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b6e8d7f69d00f40466dd9ac7979285d70536a1ba64e3fa5ca1619325faa4ca1"
Oct 31 01:20:45.417553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.417521    8029 scope.go:122] "RemoveContainer" containerID="a27fede57469d22d738df63507de04c361ab297e3ce12ae575b4a6ee80ebd6ee"
Oct 31 01:20:45.427050 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.427028    8029 scope.go:122] "RemoveContainer" containerID="1c5ab74ab75f43e37e877b911b886b5c7b4bd0c8e42c7d60f394006753526a98"
Oct 31 01:20:45.435872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.435849    8029 scope.go:122] "RemoveContainer" containerID="5091e41d389d176235945ca3c80a2b9341a283429b05fed0610e870393350a48"
Oct 31 01:20:45.443889 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.443872    8029 scope.go:122] "RemoveContainer" containerID="342d6f1e05db119c536e2c0bf250f2d53b30304c178960224fb857d9353947ce"
Oct 31 01:20:45.452015 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.451996    8029 scope.go:122] "RemoveContainer" containerID="a59b0cb8b5b65ba96a3c01dc2473c68c7091c25bf4593a674daa10aadea2287a"
Oct 31 01:20:45.461117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.461094    8029 scope.go:122] "RemoveContainer" containerID="da1e693c4a7ddf47f5f61c6a40986b0446a258a4e5691d80101e02ef10e61398"
Oct 31 01:20:45.469388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.469368    8029 scope.go:122] "RemoveContainer" containerID="44feb80a579528badbc3fc510b8476544c6f2658ab3508b6db9cabe3d669b350"
Oct 31 01:20:45.477586 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.477568    8029 scope.go:122] "RemoveContainer" containerID="f0dfc0b6f73ec515eea6bfda0e290b20359a82964b5056517c07b4d48b8998bf"
Oct 31 01:20:45.485679 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.485655    8029 scope.go:122] "RemoveContainer" containerID="bb014f206d2a2f852b83f397497405222ac9315b5f2c1668130ed45862fe30c6"
Oct 31 01:20:45.497860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.497815    8029 scope.go:122] "RemoveContainer" containerID="7c184dd284fbb5eb1a5d006ae2216c46d00e354acc164dd4bd279ad921c7f2b9"
Oct 31 01:20:45.508983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.508965    8029 scope.go:122] "RemoveContainer" containerID="f50eb25b879a59fd6f8d514cfa6e8ed2c2edd8d59ee40e62977ec42de95c10eb"
Oct 31 01:20:45.517526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.517503    8029 scope.go:122] "RemoveContainer" containerID="f9d577698f4ed5f8a4514d2d095952b6463b22eaf5d5f130e00aba74a4506a86"
Oct 31 01:20:45.528480 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.528463    8029 scope.go:122] "RemoveContainer" containerID="e080cd0d4ad49af0f4927e0966b3c12421cf74bcfa84acec1d2eb729fbc77603"
Oct 31 01:20:45.539076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.539059    8029 scope.go:122] "RemoveContainer" containerID="deadab4dcfdbfdbe010ada28d626878b5c4d9c9aa2618a87d782819672fa259f"
Oct 31 01:20:45.548075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.548053    8029 scope.go:122] "RemoveContainer" containerID="b71c5db2c4961d1d18b6166323ef460321c91422b9fe7e9b2969f6215b121916"
Oct 31 01:20:45.556174 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.556151    8029 scope.go:122] "RemoveContainer" containerID="96907b566a5a1c3e13a27c8849681c1eacf1f389a2741a76a3424f2d753882c2"
Oct 31 01:20:45.576940 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.576910    8029 scope.go:122] "RemoveContainer" containerID="888eedd6ef34d09646982fd0f582b356bade2e93835fe3061bb4091454e016e2"
Oct 31 01:20:45.587751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.587727    8029 scope.go:122] "RemoveContainer" containerID="981ac009644edec54bf3ccd8d1a34528cf5af102b1244749ba3a0ee104f0202c"
Oct 31 01:20:45.597942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.597913    8029 scope.go:122] "RemoveContainer" containerID="f86b23bf04e9b56b970f9beb9c572b6c61dd7fd5da91ac3ddbdd897717d31179"
Oct 31 01:20:45.606736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.606719    8029 scope.go:122] "RemoveContainer" containerID="b5395b605e857cc7f99fd4d41ffedc49f9268269a2402632d3db287493179062"
Oct 31 01:20:45.615873 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.615856    8029 scope.go:122] "RemoveContainer" containerID="3df794e5fd30905f5d4491ee15d56faafc9677eef109e6951341acb54395f6d5"
Oct 31 01:20:45.623585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:45.623568    8029 scope.go:122] "RemoveContainer" containerID="3ac34bd6b7940bce6155838da06254699d5b4b116656b923892371ba25e28ca1"
Oct 31 01:20:49.443697 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:20:49.443589    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:20:58.661218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.661179    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"]
Oct 31 01:20:58.661827 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.661806    8029 cpu_manager.go:414] "containerMap: removing container" podUID="73bcc3c2-b758-42f2-9625-3f6c94162cb2" containerName="revoke-keys"
Oct 31 01:20:58.661922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.661829    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73bcc3c2-b758-42f2-9625-3f6c94162cb2" containerName="revoke-keys"
Oct 31 01:20:58.661980 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.661965    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="73bcc3c2-b758-42f2-9625-3f6c94162cb2" containerName="revoke-keys"
Oct 31 01:20:58.665023 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.665003    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.683052 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.683025    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-2753ec\""
Oct 31 01:20:58.691698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.691672    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"]
Oct 31 01:20:58.704889 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.704866    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-cf36fd16-dockercfg-swv66\""
Oct 31 01:20:58.774664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.774631    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr269\" (UniqueName: \"kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269\") pod \"payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.774815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.774687    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.817267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.817240    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"]
Oct 31 01:20:58.821427 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.821399    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:58.849842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.849808    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"]
Oct 31 01:20:58.876134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.876101    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76sh4\" (UniqueName: \"kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4\") pod \"payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:58.876293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.876209    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr269\" (UniqueName: \"kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269\") pod \"payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.876293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.876259    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.876365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.876293    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:58.876954 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.876936    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.902776 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.902739    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr269\" (UniqueName: \"kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269\") pod \"payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.977101 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.977033    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76sh4\" (UniqueName: \"kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4\") pod \"payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:58.977242 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.977111    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:58.977324 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.977305    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:58.977812 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.977793    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:58.991593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:58.991564    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76sh4\" (UniqueName: \"kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4\") pod \"payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:59.138765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.138726    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:59.265992 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.265916    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"]
Oct 31 01:20:59.269276 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:20:59.269249    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2038e5b1_a904_4a2f_8e1a_db906e4f91df.slice/crio-4ac2d9cfe48ead657cee70b8dce553a2c4d28d55248ccda73694f836854b2ef4 WatchSource:0}: Error finding container 4ac2d9cfe48ead657cee70b8dce553a2c4d28d55248ccda73694f836854b2ef4: Status 404 returned error can't find the container with id 4ac2d9cfe48ead657cee70b8dce553a2c4d28d55248ccda73694f836854b2ef4
Oct 31 01:20:59.425693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.425654    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"]
Oct 31 01:20:59.427958 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:20:59.427931    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8b68e6_f334_41f7_b1dd_b774aabe7f82.slice/crio-bde0e276d3fa9736e135b81ebc849ede06d3c0213f0ebb4e9549813d91a25a73 WatchSource:0}: Error finding container bde0e276d3fa9736e135b81ebc849ede06d3c0213f0ebb4e9549813d91a25a73: Status 404 returned error can't find the container with id bde0e276d3fa9736e135b81ebc849ede06d3c0213f0ebb4e9549813d91a25a73
Oct 31 01:20:59.515989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.515940    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" event={"ID":"fb8b68e6-f334-41f7-b1dd-b774aabe7f82","Type":"ContainerStarted","Data":"78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6"}
Oct 31 01:20:59.516228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.516010    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" event={"ID":"fb8b68e6-f334-41f7-b1dd-b774aabe7f82","Type":"ContainerStarted","Data":"bde0e276d3fa9736e135b81ebc849ede06d3c0213f0ebb4e9549813d91a25a73"}
Oct 31 01:20:59.516318 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.516255    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:20:59.518373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.518339    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" event={"ID":"2038e5b1-a904-4a2f-8e1a-db906e4f91df","Type":"ContainerStarted","Data":"005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8"}
Oct 31 01:20:59.518509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.518401    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" event={"ID":"2038e5b1-a904-4a2f-8e1a-db906e4f91df","Type":"ContainerStarted","Data":"4ac2d9cfe48ead657cee70b8dce553a2c4d28d55248ccda73694f836854b2ef4"}
Oct 31 01:20:59.518509 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.518434    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:20:59.585888 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.585831    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" podStartSLOduration=1.585817351 podStartE2EDuration="1.585817351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:59.584809986 +0000 UTC m=+1456.571921966" watchObservedRunningTime="2026-10-31 01:20:59.585817351 +0000 UTC m=+1456.572929324"
Oct 31 01:20:59.682086 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:20:59.682036    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" podStartSLOduration=1.682021764 podStartE2EDuration="1.682021764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:20:59.680457282 +0000 UTC m=+1456.667569260" watchObservedRunningTime="2026-10-31 01:20:59.682021764 +0000 UTC m=+1456.669133741"
Oct 31 01:21:10.527136 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:10.527108    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:21:10.527634 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:10.527175    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:21:19.442667 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:21:19.442627    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:21:43.444083 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:43.444048    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:21:43.444744 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:43.444395    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:21:44.036916 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.036881    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"]
Oct 31 01:21:44.040764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.040738    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.047257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.047236    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:21:44.047257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.047238    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:21:44.047473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.047384    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:21:44.047473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.047405    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:21:44.056787 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.056766    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"]
Oct 31 01:21:44.142387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.142342    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpdj9\" (UniqueName: \"kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.142387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.142388    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.142673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.142490    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.243484 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.243442    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpdj9\" (UniqueName: \"kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.243484 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.243485    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.243765 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.243521    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.244208 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.244184    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.246216 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.246185    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.252075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.252048    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpdj9\" (UniqueName: \"kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9\") pod \"maas-api-revoke-sub-991e3f8a-ttqmm\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.354704 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.354677    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:44.636169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.636143    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"]
Oct 31 01:21:44.639081 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:21:44.639053    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a22103_042b_47c1_93a7_6a76685eede3.slice/crio-150fa50ee4d7b6c325c30dc98a22df11f9bc804d3553b4b6ce3df11e3e7d7b71 WatchSource:0}: Error finding container 150fa50ee4d7b6c325c30dc98a22df11f9bc804d3553b4b6ce3df11e3e7d7b71: Status 404 returned error can't find the container with id 150fa50ee4d7b6c325c30dc98a22df11f9bc804d3553b4b6ce3df11e3e7d7b71
Oct 31 01:21:44.751378 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:44.751346    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm" event={"ID":"99a22103-042b-47c1-93a7-6a76685eede3","Type":"ContainerStarted","Data":"150fa50ee4d7b6c325c30dc98a22df11f9bc804d3553b4b6ce3df11e3e7d7b71"}
Oct 31 01:21:45.744304 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.744223    8029 scope.go:122] "RemoveContainer" containerID="389265b61dc346afa7f001da822ac0c828ade0ac38f81aa6ca216d67a445426f"
Oct 31 01:21:45.753741 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.753718    8029 scope.go:122] "RemoveContainer" containerID="db727b97c1f61c58b271d500a27492c506ee00233b39cca09008050cd8ca4c07"
Oct 31 01:21:45.760687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.760623    8029 generic.go:356] "Generic (PLEG): container finished" podID="99a22103-042b-47c1-93a7-6a76685eede3" containerID="7621fecf6df443e92568dbbe69afc2314494b549562b240f78d7e0cf0343a4c8" exitCode=0
Oct 31 01:21:45.760826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.760698    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm" event={"ID":"99a22103-042b-47c1-93a7-6a76685eede3","Type":"ContainerDied","Data":"7621fecf6df443e92568dbbe69afc2314494b549562b240f78d7e0cf0343a4c8"}
Oct 31 01:21:45.763469 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.763449    8029 scope.go:122] "RemoveContainer" containerID="af5221870324eff01a072beab270915e4fd216533b5dcbf2df6f8fcacd4990db"
Oct 31 01:21:45.772059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.772040    8029 scope.go:122] "RemoveContainer" containerID="a2065de1bde97bcc31a85bcd171ebcd657bbe8fcabe30fab703414541bfdb3e3"
Oct 31 01:21:45.781638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:45.781590    8029 scope.go:122] "RemoveContainer" containerID="55cc2ef78c8a0f2b6f0d1087c1cf68730aa693b8422a4b24e992267dfbd764e1"
Oct 31 01:21:46.907835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.907809    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:46.970794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.970765    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca\") pod \"99a22103-042b-47c1-93a7-6a76685eede3\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") "
Oct 31 01:21:46.970992 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.970835    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca\") pod \"99a22103-042b-47c1-93a7-6a76685eede3\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") "
Oct 31 01:21:46.970992 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.970936    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9\" (UniqueName: \"kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9\") pod \"99a22103-042b-47c1-93a7-6a76685eede3\" (UID: \"99a22103-042b-47c1-93a7-6a76685eede3\") "
Oct 31 01:21:46.971210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.971179    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca" pod "99a22103-042b-47c1-93a7-6a76685eede3" (UID: "99a22103-042b-47c1-93a7-6a76685eede3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:21:46.971467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.971376    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:46.973228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.973196    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca" pod "99a22103-042b-47c1-93a7-6a76685eede3" (UID: "99a22103-042b-47c1-93a7-6a76685eede3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:21:46.973334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:46.973316    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9" pod "99a22103-042b-47c1-93a7-6a76685eede3" (UID: "99a22103-042b-47c1-93a7-6a76685eede3"). InnerVolumeSpecName "kube-api-access-lpdj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:21:47.072698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:47.072634    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a22103-042b-47c1-93a7-6a76685eede3-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:47.072698 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:47.072660    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpdj9\" (UniqueName: \"kubernetes.io/projected/99a22103-042b-47c1-93a7-6a76685eede3-kube-api-access-lpdj9\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:47.773248 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:47.773217    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"
Oct 31 01:21:47.773446 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:47.773207    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm" event={"ID":"99a22103-042b-47c1-93a7-6a76685eede3","Type":"ContainerDied","Data":"150fa50ee4d7b6c325c30dc98a22df11f9bc804d3553b4b6ce3df11e3e7d7b71"}
Oct 31 01:21:47.773446 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:47.773352    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="150fa50ee4d7b6c325c30dc98a22df11f9bc804d3553b4b6ce3df11e3e7d7b71"
Oct 31 01:21:49.447031 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:21:49.446987    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:21:55.020350 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.020311    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"]
Oct 31 01:21:55.020778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.020747    8029 cpu_manager.go:414] "containerMap: removing container" podUID="99a22103-042b-47c1-93a7-6a76685eede3" containerName="revoke-keys"
Oct 31 01:21:55.020778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.020759    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99a22103-042b-47c1-93a7-6a76685eede3" containerName="revoke-keys"
Oct 31 01:21:55.020871 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.020860    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="99a22103-042b-47c1-93a7-6a76685eede3" containerName="revoke-keys"
Oct 31 01:21:55.024383 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.024363    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.030475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.030454    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:21:55.030697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.030678    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:21:55.030836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.030797    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:21:55.030836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.030833    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:21:55.031403 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.031375    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"]
Oct 31 01:21:55.153031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.152993    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.153232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.153048    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.153331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.153229    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq769\" (UniqueName: \"kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.254117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.254072    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq769\" (UniqueName: \"kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.254335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.254165    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.254335 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.254203    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.254876 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.254855    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.256801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.256771    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.262095 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.262070    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq769\" (UniqueName: \"kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769\") pod \"maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.339877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.339775    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:55.620531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.620497    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"]
Oct 31 01:21:55.623754 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:21:55.623726    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2701295b_2e03_4264_94f5_62c4ef338ffa.slice/crio-5ba1dbaf5c8bf4369673beb865c40b10f3ac1573f4bad1957e96128a6b8a2a8b WatchSource:0}: Error finding container 5ba1dbaf5c8bf4369673beb865c40b10f3ac1573f4bad1957e96128a6b8a2a8b: Status 404 returned error can't find the container with id 5ba1dbaf5c8bf4369673beb865c40b10f3ac1573f4bad1957e96128a6b8a2a8b
Oct 31 01:21:55.814504 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:55.814475    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz" event={"ID":"2701295b-2e03-4264-94f5-62c4ef338ffa","Type":"ContainerStarted","Data":"5ba1dbaf5c8bf4369673beb865c40b10f3ac1573f4bad1957e96128a6b8a2a8b"}
Oct 31 01:21:56.820926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:56.820885    8029 generic.go:356] "Generic (PLEG): container finished" podID="2701295b-2e03-4264-94f5-62c4ef338ffa" containerID="26de1dd3e293b8a130886ed3aa11d796e45e767e26ae42fc32bf105eb4414f12" exitCode=0
Oct 31 01:21:56.821362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:56.820969    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz" event={"ID":"2701295b-2e03-4264-94f5-62c4ef338ffa","Type":"ContainerDied","Data":"26de1dd3e293b8a130886ed3aa11d796e45e767e26ae42fc32bf105eb4414f12"}
Oct 31 01:21:57.964441 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:57.964417    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:58.087138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.087061    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca\") pod \"2701295b-2e03-4264-94f5-62c4ef338ffa\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") "
Oct 31 01:21:58.087138 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.087120    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769\" (UniqueName: \"kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769\") pod \"2701295b-2e03-4264-94f5-62c4ef338ffa\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") "
Oct 31 01:21:58.087330 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.087240    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca\") pod \"2701295b-2e03-4264-94f5-62c4ef338ffa\" (UID: \"2701295b-2e03-4264-94f5-62c4ef338ffa\") "
Oct 31 01:21:58.087517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.087488    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca" pod "2701295b-2e03-4264-94f5-62c4ef338ffa" (UID: "2701295b-2e03-4264-94f5-62c4ef338ffa"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:21:58.089462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.089431    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca" pod "2701295b-2e03-4264-94f5-62c4ef338ffa" (UID: "2701295b-2e03-4264-94f5-62c4ef338ffa"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:21:58.089462 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.089446    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769" pod "2701295b-2e03-4264-94f5-62c4ef338ffa" (UID: "2701295b-2e03-4264-94f5-62c4ef338ffa"). InnerVolumeSpecName "kube-api-access-pq769". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:21:58.188190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.188154    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq769\" (UniqueName: \"kubernetes.io/projected/2701295b-2e03-4264-94f5-62c4ef338ffa-kube-api-access-pq769\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:58.188190 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.188185    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:58.188391 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.188200    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2701295b-2e03-4264-94f5-62c4ef338ffa-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:21:58.832686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.832659    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"
Oct 31 01:21:58.832897 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.832692    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz" event={"ID":"2701295b-2e03-4264-94f5-62c4ef338ffa","Type":"ContainerDied","Data":"5ba1dbaf5c8bf4369673beb865c40b10f3ac1573f4bad1957e96128a6b8a2a8b"}
Oct 31 01:21:58.832897 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:21:58.832740    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ba1dbaf5c8bf4369673beb865c40b10f3ac1573f4bad1957e96128a6b8a2a8b"
Oct 31 01:22:05.163344 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:05.163306    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"]
Oct 31 01:22:05.175761 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:05.175730    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"]
Oct 31 01:22:06.081951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:06.081900    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"]
Oct 31 01:22:06.086427 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:06.086398    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-2753ec-r8wfz"]
Oct 31 01:22:07.182466 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:07.182437    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2701295b-2e03-4264-94f5-62c4ef338ffa" path="/var/lib/kubelet/pods/2701295b-2e03-4264-94f5-62c4ef338ffa/volumes"
Oct 31 01:22:10.164196 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:10.164139    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerName="payload-processing" containerID="cri-o://78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6" gracePeriod=25
Oct 31 01:22:10.176082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:10.176050    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerName="payload-pre-processing" containerID="cri-o://005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8" gracePeriod=25
Oct 31 01:22:10.525726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:10.525541    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:22:10.525726 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:10.525557    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:22:14.181770 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.181736    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"]
Oct 31 01:22:14.186068 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.186042    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-4xxlf"]
Oct 31 01:22:14.204631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.204590    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"]
Oct 31 01:22:14.205090 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.205078    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2701295b-2e03-4264-94f5-62c4ef338ffa" containerName="revoke-keys"
Oct 31 01:22:14.205154 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.205092    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2701295b-2e03-4264-94f5-62c4ef338ffa" containerName="revoke-keys"
Oct 31 01:22:14.205193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.205165    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2701295b-2e03-4264-94f5-62c4ef338ffa" containerName="revoke-keys"
Oct 31 01:22:14.208399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.208383    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.211904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.211879    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:22:14.212047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.211916    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:22:14.212047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.212010    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:22:14.214511 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.214480    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:22:14.217245 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.217221    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"]
Oct 31 01:22:14.253926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.253900    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6zqp\" (UniqueName: \"kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.254085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.253977    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.254085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.254054    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.354818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.354786    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.355011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.354842    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.355011 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.354906    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6zqp\" (UniqueName: \"kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.355506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.355483    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.357289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.357270    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.362977 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.362948    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6zqp\" (UniqueName: \"kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp\") pod \"maas-api-revoke-sub-3d4156f1-652nq\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.522682 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.522583    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:14.794841 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.794819    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"]
Oct 31 01:22:14.797545 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:22:14.797515    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98437962_3278_4948_8b48_c986dce13b9a.slice/crio-bb97b25ed3f8380b1f53f67137401a0e1db757cd8e0fb2e555ab8f771007c417 WatchSource:0}: Error finding container bb97b25ed3f8380b1f53f67137401a0e1db757cd8e0fb2e555ab8f771007c417: Status 404 returned error can't find the container with id bb97b25ed3f8380b1f53f67137401a0e1db757cd8e0fb2e555ab8f771007c417
Oct 31 01:22:14.914449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:14.914413    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq" event={"ID":"98437962-3278-4948-8b48-c986dce13b9a","Type":"ContainerStarted","Data":"bb97b25ed3f8380b1f53f67137401a0e1db757cd8e0fb2e555ab8f771007c417"}
Oct 31 01:22:15.183657 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:15.183623    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73bcc3c2-b758-42f2-9625-3f6c94162cb2" path="/var/lib/kubelet/pods/73bcc3c2-b758-42f2-9625-3f6c94162cb2/volumes"
Oct 31 01:22:15.920982 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:15.920948    8029 generic.go:356] "Generic (PLEG): container finished" podID="98437962-3278-4948-8b48-c986dce13b9a" containerID="8a3e24523fba981a7c09d500cf72d878303558a6d525649b872363ea54ab3913" exitCode=0
Oct 31 01:22:15.921164 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:15.921005    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq" event={"ID":"98437962-3278-4948-8b48-c986dce13b9a","Type":"ContainerDied","Data":"8a3e24523fba981a7c09d500cf72d878303558a6d525649b872363ea54ab3913"}
Oct 31 01:22:17.074075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.074051    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:17.183652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.183511    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca\") pod \"98437962-3278-4948-8b48-c986dce13b9a\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") "
Oct 31 01:22:17.183652 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.183575    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca\") pod \"98437962-3278-4948-8b48-c986dce13b9a\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") "
Oct 31 01:22:17.183890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.183673    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp\" (UniqueName: \"kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp\") pod \"98437962-3278-4948-8b48-c986dce13b9a\" (UID: \"98437962-3278-4948-8b48-c986dce13b9a\") "
Oct 31 01:22:17.183890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.183881    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca" pod "98437962-3278-4948-8b48-c986dce13b9a" (UID: "98437962-3278-4948-8b48-c986dce13b9a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:22:17.184141 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.184104    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/98437962-3278-4948-8b48-c986dce13b9a-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:17.185999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.185973    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca" pod "98437962-3278-4948-8b48-c986dce13b9a" (UID: "98437962-3278-4948-8b48-c986dce13b9a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:22:17.186109 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.186025    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp" pod "98437962-3278-4948-8b48-c986dce13b9a" (UID: "98437962-3278-4948-8b48-c986dce13b9a"). InnerVolumeSpecName "kube-api-access-l6zqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:22:17.285250 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.285207    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/98437962-3278-4948-8b48-c986dce13b9a-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:17.285250 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.285245    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6zqp\" (UniqueName: \"kubernetes.io/projected/98437962-3278-4948-8b48-c986dce13b9a-kube-api-access-l6zqp\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:17.932284 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.932258    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"
Oct 31 01:22:17.932483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.932299    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq" event={"ID":"98437962-3278-4948-8b48-c986dce13b9a","Type":"ContainerDied","Data":"bb97b25ed3f8380b1f53f67137401a0e1db757cd8e0fb2e555ab8f771007c417"}
Oct 31 01:22:17.932483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:17.932347    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb97b25ed3f8380b1f53f67137401a0e1db757cd8e0fb2e555ab8f771007c417"
Oct 31 01:22:19.443163 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:22:19.443080    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:22:20.525920 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:20.525884    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:22:20.526301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:20.525887    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:22:30.425595 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.425540    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:22:30.455534 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.455509    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:22:30.544708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.544675    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4\" (UniqueName: \"kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4\") pod \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") "
Oct 31 01:22:30.544903 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.544785    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume\") pod \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") "
Oct 31 01:22:30.544903 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.544818    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume\") pod \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\" (UID: \"fb8b68e6-f334-41f7-b1dd-b774aabe7f82\") "
Oct 31 01:22:30.544985 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.544917    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269\" (UniqueName: \"kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269\") pod \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\" (UID: \"2038e5b1-a904-4a2f-8e1a-db906e4f91df\") "
Oct 31 01:22:30.545234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.545206    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume" pod "2038e5b1-a904-4a2f-8e1a-db906e4f91df" (UID: "2038e5b1-a904-4a2f-8e1a-db906e4f91df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:22:30.545234 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.545208    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume" pod "fb8b68e6-f334-41f7-b1dd-b774aabe7f82" (UID: "fb8b68e6-f334-41f7-b1dd-b774aabe7f82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:22:30.545405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.545298    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2038e5b1-a904-4a2f-8e1a-db906e4f91df-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:30.545405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.545318    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:30.547065 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.547043    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269" pod "2038e5b1-a904-4a2f-8e1a-db906e4f91df" (UID: "2038e5b1-a904-4a2f-8e1a-db906e4f91df"). InnerVolumeSpecName "kube-api-access-wr269". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:22:30.547158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.547074    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4" pod "fb8b68e6-f334-41f7-b1dd-b774aabe7f82" (UID: "fb8b68e6-f334-41f7-b1dd-b774aabe7f82"). InnerVolumeSpecName "kube-api-access-76sh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:22:30.646692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.646651    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr269\" (UniqueName: \"kubernetes.io/projected/2038e5b1-a904-4a2f-8e1a-db906e4f91df-kube-api-access-wr269\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:30.646692 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.646683    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76sh4\" (UniqueName: \"kubernetes.io/projected/fb8b68e6-f334-41f7-b1dd-b774aabe7f82-kube-api-access-76sh4\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:22:30.998315 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.998281    8029 generic.go:356] "Generic (PLEG): container finished" podID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerID="78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6" exitCode=0
Oct 31 01:22:30.998524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.998325    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"
Oct 31 01:22:30.998524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.998364    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" event={"ID":"fb8b68e6-f334-41f7-b1dd-b774aabe7f82","Type":"ContainerDied","Data":"78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6"}
Oct 31 01:22:30.998524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.998430    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw" event={"ID":"fb8b68e6-f334-41f7-b1dd-b774aabe7f82","Type":"ContainerDied","Data":"bde0e276d3fa9736e135b81ebc849ede06d3c0213f0ebb4e9549813d91a25a73"}
Oct 31 01:22:30.998524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:30.998463    8029 scope.go:122] "RemoveContainer" containerID="78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6"
Oct 31 01:22:31.002632 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.002595    8029 generic.go:356] "Generic (PLEG): container finished" podID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerID="005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8" exitCode=0
Oct 31 01:22:31.002720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.002668    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"
Oct 31 01:22:31.002720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.002693    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" event={"ID":"2038e5b1-a904-4a2f-8e1a-db906e4f91df","Type":"ContainerDied","Data":"005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8"}
Oct 31 01:22:31.002794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.002735    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9" event={"ID":"2038e5b1-a904-4a2f-8e1a-db906e4f91df","Type":"ContainerDied","Data":"4ac2d9cfe48ead657cee70b8dce553a2c4d28d55248ccda73694f836854b2ef4"}
Oct 31 01:22:31.011716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.011649    8029 scope.go:122] "RemoveContainer" containerID="78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6"
Oct 31 01:22:31.012006 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:22:31.011979    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6\": container with ID starting with 78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6 not found: ID does not exist" containerID="78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6"
Oct 31 01:22:31.012117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.012014    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6"} err="failed to get container status \"78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6\": rpc error: code = NotFound desc = could not find container \"78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6\": container with ID starting with 78949fdcaec5d5d208f4b6bc6a33ab2fe30a52798ac382a6e25565479dccabf6 not found: ID does not exist"
Oct 31 01:22:31.012117 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.012045    8029 scope.go:122] "RemoveContainer" containerID="005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8"
Oct 31 01:22:31.020307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.020289    8029 scope.go:122] "RemoveContainer" containerID="005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8"
Oct 31 01:22:31.020668 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:22:31.020641    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8\": container with ID starting with 005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8 not found: ID does not exist" containerID="005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8"
Oct 31 01:22:31.021218 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.020676    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8"} err="failed to get container status \"005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8\": rpc error: code = NotFound desc = could not find container \"005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8\": container with ID starting with 005fab846392604334fff9912ca3f253faedba25587afca24f32be3cfb62bcd8 not found: ID does not exist"
Oct 31 01:22:31.022703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.022684    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"]
Oct 31 01:22:31.027684 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.027664    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-2753ec-6c65b65469gjdw"]
Oct 31 01:22:31.042373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.042336    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"]
Oct 31 01:22:31.046380 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.046357    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-2753ec-5874cn2rs9"]
Oct 31 01:22:31.190365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.190330    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" path="/var/lib/kubelet/pods/2038e5b1-a904-4a2f-8e1a-db906e4f91df/volumes"
Oct 31 01:22:31.190814 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:22:31.190794    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" path="/var/lib/kubelet/pods/fb8b68e6-f334-41f7-b1dd-b774aabe7f82/volumes"
Oct 31 01:22:49.442513 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:22:49.442476    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:23:19.442367 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:23:19.442328    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:23:49.442907 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:23:49.442822    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:23:57.513975 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.513936    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"]
Oct 31 01:23:57.514381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514361    8029 cpu_manager.go:414] "containerMap: removing container" podUID="98437962-3278-4948-8b48-c986dce13b9a" containerName="revoke-keys"
Oct 31 01:23:57.514381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514375    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98437962-3278-4948-8b48-c986dce13b9a" containerName="revoke-keys"
Oct 31 01:23:57.514466 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514385    8029 cpu_manager.go:414] "containerMap: removing container" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerName="payload-processing"
Oct 31 01:23:57.514466 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514392    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerName="payload-processing"
Oct 31 01:23:57.514466 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514407    8029 cpu_manager.go:414] "containerMap: removing container" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerName="payload-pre-processing"
Oct 31 01:23:57.514466 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514412    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerName="payload-pre-processing"
Oct 31 01:23:57.514636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514495    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="98437962-3278-4948-8b48-c986dce13b9a" containerName="revoke-keys"
Oct 31 01:23:57.514636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514503    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb8b68e6-f334-41f7-b1dd-b774aabe7f82" containerName="payload-processing"
Oct 31 01:23:57.514636 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.514512    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2038e5b1-a904-4a2f-8e1a-db906e4f91df" containerName="payload-pre-processing"
Oct 31 01:23:57.517893 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.517871    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.524558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.524527    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:23:57.524558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.524531    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:23:57.524780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.524575    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:23:57.524780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.524637    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:23:57.525800 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.525434    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"]
Oct 31 01:23:57.563519 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.563489    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.563662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.563569    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.563709 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.563665    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp86w\" (UniqueName: \"kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.664483 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.664451    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.664656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.664506    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp86w\" (UniqueName: \"kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.664656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.664556    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.665160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.665142    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.666879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.666862    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.673191 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.673169    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp86w\" (UniqueName: \"kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w\") pod \"maas-api-revoke-sub-d576e68b-tknrv\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:57.832584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:57.832513    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:23:58.107074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:58.107045    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"]
Oct 31 01:23:58.111639 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:23:58.111082    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85768033_0b00_447c_9d8e_6779092ba3c6.slice/crio-b97684472c085b253a24fdcc56b23009ca260a05664f9efa71ecef91aecd45dc WatchSource:0}: Error finding container b97684472c085b253a24fdcc56b23009ca260a05664f9efa71ecef91aecd45dc: Status 404 returned error can't find the container with id b97684472c085b253a24fdcc56b23009ca260a05664f9efa71ecef91aecd45dc
Oct 31 01:23:58.438871 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:58.438841    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv" event={"ID":"85768033-0b00-447c-9d8e-6779092ba3c6","Type":"ContainerStarted","Data":"8bd86fa8052669e8f55be26caa0ae95e205bb49a30c465479cdc6fe307ebd64f"}
Oct 31 01:23:58.439001 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:58.438918    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv" event={"ID":"85768033-0b00-447c-9d8e-6779092ba3c6","Type":"ContainerStarted","Data":"b97684472c085b253a24fdcc56b23009ca260a05664f9efa71ecef91aecd45dc"}
Oct 31 01:23:58.456075 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:58.456008    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv" podStartSLOduration=1.455995963 podStartE2EDuration="1.455995963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:23:58.455126328 +0000 UTC m=+1635.442238304" watchObservedRunningTime="2026-10-31 01:23:58.455995963 +0000 UTC m=+1635.443107953"
Oct 31 01:23:59.445588 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:59.445554    8029 generic.go:356] "Generic (PLEG): container finished" podID="85768033-0b00-447c-9d8e-6779092ba3c6" containerID="8bd86fa8052669e8f55be26caa0ae95e205bb49a30c465479cdc6fe307ebd64f" exitCode=0
Oct 31 01:23:59.446008 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:23:59.445644    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv" event={"ID":"85768033-0b00-447c-9d8e-6779092ba3c6","Type":"ContainerDied","Data":"8bd86fa8052669e8f55be26caa0ae95e205bb49a30c465479cdc6fe307ebd64f"}
Oct 31 01:24:00.589880 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.589856    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:24:00.694179 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.694139    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w\" (UniqueName: \"kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w\") pod \"85768033-0b00-447c-9d8e-6779092ba3c6\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") "
Oct 31 01:24:00.694385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.694198    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca\") pod \"85768033-0b00-447c-9d8e-6779092ba3c6\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") "
Oct 31 01:24:00.694385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.694233    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca\") pod \"85768033-0b00-447c-9d8e-6779092ba3c6\" (UID: \"85768033-0b00-447c-9d8e-6779092ba3c6\") "
Oct 31 01:24:00.694687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.694664    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca" pod "85768033-0b00-447c-9d8e-6779092ba3c6" (UID: "85768033-0b00-447c-9d8e-6779092ba3c6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:24:00.696529 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.696498    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca" pod "85768033-0b00-447c-9d8e-6779092ba3c6" (UID: "85768033-0b00-447c-9d8e-6779092ba3c6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:24:00.696662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.696570    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w" pod "85768033-0b00-447c-9d8e-6779092ba3c6" (UID: "85768033-0b00-447c-9d8e-6779092ba3c6"). InnerVolumeSpecName "kube-api-access-zp86w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:24:00.795593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.795547    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp86w\" (UniqueName: \"kubernetes.io/projected/85768033-0b00-447c-9d8e-6779092ba3c6-kube-api-access-zp86w\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:24:00.795593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.795585    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:24:00.795593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:00.795611    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85768033-0b00-447c-9d8e-6779092ba3c6-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:24:01.457235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:01.457156    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"
Oct 31 01:24:01.457235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:01.457173    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv" event={"ID":"85768033-0b00-447c-9d8e-6779092ba3c6","Type":"ContainerDied","Data":"b97684472c085b253a24fdcc56b23009ca260a05664f9efa71ecef91aecd45dc"}
Oct 31 01:24:01.457235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:01.457219    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b97684472c085b253a24fdcc56b23009ca260a05664f9efa71ecef91aecd45dc"
Oct 31 01:24:16.363417 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.363381    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"]
Oct 31 01:24:16.363860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.363847    8029 cpu_manager.go:414] "containerMap: removing container" podUID="85768033-0b00-447c-9d8e-6779092ba3c6" containerName="revoke-keys"
Oct 31 01:24:16.363901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.363860    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85768033-0b00-447c-9d8e-6779092ba3c6" containerName="revoke-keys"
Oct 31 01:24:16.363941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.363929    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="85768033-0b00-447c-9d8e-6779092ba3c6" containerName="revoke-keys"
Oct 31 01:24:16.366984 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.366964    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.373503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.373469    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:24:16.373990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.373752    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:24:16.373990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.373817    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:24:16.373990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.373887    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:24:16.374916 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.374886    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"]
Oct 31 01:24:16.480002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.479965    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.480180 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.480086    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.480180 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.480138    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcfk6\" (UniqueName: \"kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.580874 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.580839    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.580874 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.580890    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcfk6\" (UniqueName: \"kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.581100 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.580962    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.581527 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.581495    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.583419 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.583399    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.589055 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.589031    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcfk6\" (UniqueName: \"kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6\") pod \"maas-api-revoke-sub-d2bc93bc-v7876\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.681881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.681783    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:16.954039 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:16.954006    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"]
Oct 31 01:24:16.956676 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:24:16.956651    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489f90fc_ab63_4378_8e94_ca46780e7d59.slice/crio-0ed6ca273a2448b03b954550c321486dcb106c829ee39edae25d1c20b358c2c4 WatchSource:0}: Error finding container 0ed6ca273a2448b03b954550c321486dcb106c829ee39edae25d1c20b358c2c4: Status 404 returned error can't find the container with id 0ed6ca273a2448b03b954550c321486dcb106c829ee39edae25d1c20b358c2c4
Oct 31 01:24:17.573207 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:17.573104    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876" event={"ID":"489f90fc-ab63-4378-8e94-ca46780e7d59","Type":"ContainerStarted","Data":"68f34c1c85e028e73ae00b33c533a8ba6581302d0b5e6ec05f14d03e372ecdfe"}
Oct 31 01:24:17.573207 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:17.573165    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876" event={"ID":"489f90fc-ab63-4378-8e94-ca46780e7d59","Type":"ContainerStarted","Data":"0ed6ca273a2448b03b954550c321486dcb106c829ee39edae25d1c20b358c2c4"}
Oct 31 01:24:17.589183 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:17.589128    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876" podStartSLOduration=1.589114608 podStartE2EDuration="1.589114608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:24:17.587642336 +0000 UTC m=+1654.574754314" watchObservedRunningTime="2026-10-31 01:24:17.589114608 +0000 UTC m=+1654.576226584"
Oct 31 01:24:18.579311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:18.579261    8029 generic.go:356] "Generic (PLEG): container finished" podID="489f90fc-ab63-4378-8e94-ca46780e7d59" containerID="68f34c1c85e028e73ae00b33c533a8ba6581302d0b5e6ec05f14d03e372ecdfe" exitCode=0
Oct 31 01:24:18.579709 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:18.579339    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876" event={"ID":"489f90fc-ab63-4378-8e94-ca46780e7d59","Type":"ContainerDied","Data":"68f34c1c85e028e73ae00b33c533a8ba6581302d0b5e6ec05f14d03e372ecdfe"}
Oct 31 01:24:19.443555 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:24:19.443502    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:24:19.728042 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.728010    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:19.913981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.913946    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca\") pod \"489f90fc-ab63-4378-8e94-ca46780e7d59\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") "
Oct 31 01:24:19.914170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.914033    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6\" (UniqueName: \"kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6\") pod \"489f90fc-ab63-4378-8e94-ca46780e7d59\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") "
Oct 31 01:24:19.914170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.914126    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca\") pod \"489f90fc-ab63-4378-8e94-ca46780e7d59\" (UID: \"489f90fc-ab63-4378-8e94-ca46780e7d59\") "
Oct 31 01:24:19.914356 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.914331    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca" pod "489f90fc-ab63-4378-8e94-ca46780e7d59" (UID: "489f90fc-ab63-4378-8e94-ca46780e7d59"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:24:19.914538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.914516    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:24:19.916374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.916348    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca" pod "489f90fc-ab63-4378-8e94-ca46780e7d59" (UID: "489f90fc-ab63-4378-8e94-ca46780e7d59"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:24:19.916374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:19.916355    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6" pod "489f90fc-ab63-4378-8e94-ca46780e7d59" (UID: "489f90fc-ab63-4378-8e94-ca46780e7d59"). InnerVolumeSpecName "kube-api-access-bcfk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:24:20.015222 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:20.015120    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcfk6\" (UniqueName: \"kubernetes.io/projected/489f90fc-ab63-4378-8e94-ca46780e7d59-kube-api-access-bcfk6\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:24:20.015222 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:20.015160    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/489f90fc-ab63-4378-8e94-ca46780e7d59-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:24:20.590316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:20.590281    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876" event={"ID":"489f90fc-ab63-4378-8e94-ca46780e7d59","Type":"ContainerDied","Data":"0ed6ca273a2448b03b954550c321486dcb106c829ee39edae25d1c20b358c2c4"}
Oct 31 01:24:20.590584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:20.590334    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ed6ca273a2448b03b954550c321486dcb106c829ee39edae25d1c20b358c2c4"
Oct 31 01:24:20.590584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:20.590292    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"
Oct 31 01:24:33.714227 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.714192    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"]
Oct 31 01:24:33.714672 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.714658    8029 cpu_manager.go:414] "containerMap: removing container" podUID="489f90fc-ab63-4378-8e94-ca46780e7d59" containerName="revoke-keys"
Oct 31 01:24:33.714732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.714673    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="489f90fc-ab63-4378-8e94-ca46780e7d59" containerName="revoke-keys"
Oct 31 01:24:33.714777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.714741    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="489f90fc-ab63-4378-8e94-ca46780e7d59" containerName="revoke-keys"
Oct 31 01:24:33.718033 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.718013    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.728139 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.728111    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"]
Oct 31 01:24:33.733755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.733735    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-b93c48\""
Oct 31 01:24:33.733917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.733897    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-2ff8f77e-dockercfg-pb5z5\""
Oct 31 01:24:33.749402 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.749366    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.749546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.749422    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64ns\" (UniqueName: \"kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns\") pod \"payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.755566 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.755542    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"]
Oct 31 01:24:33.759823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.759807    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:33.767244 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.767223    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"]
Oct 31 01:24:33.850538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.850503    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh6dx\" (UniqueName: \"kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx\") pod \"payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:33.850780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.850556    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.850780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.850591    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64ns\" (UniqueName: \"kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns\") pod \"payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.850780 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.850679    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:33.851506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.851482    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.859547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.859520    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64ns\" (UniqueName: \"kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns\") pod \"payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:33.951716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.951681    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh6dx\" (UniqueName: \"kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx\") pod \"payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:33.951716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.951725    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:33.952397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.952378    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:33.960329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:33.960302    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh6dx\" (UniqueName: \"kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx\") pod \"payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:34.032243 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.032159    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:34.074126 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.074094    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:34.312331 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.312146    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"]
Oct 31 01:24:34.314668 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:24:34.314640    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4603662_92a5_474b_bff3_93a7fff42db2.slice/crio-bd3eb8a36ff12e59271c9104eff2c2c5d66bdbd75561dc000dcdcc28a2c98f80 WatchSource:0}: Error finding container bd3eb8a36ff12e59271c9104eff2c2c5d66bdbd75561dc000dcdcc28a2c98f80: Status 404 returned error can't find the container with id bd3eb8a36ff12e59271c9104eff2c2c5d66bdbd75561dc000dcdcc28a2c98f80
Oct 31 01:24:34.367673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.367590    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"]
Oct 31 01:24:34.370065 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:24:34.370041    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa18ae1_2dfc_411a_8b10_ba446d948077.slice/crio-b438d8b094aba0dea4702c0ff583574112cc5bc8f6ccf34b9e58bead4b2bcbaf WatchSource:0}: Error finding container b438d8b094aba0dea4702c0ff583574112cc5bc8f6ccf34b9e58bead4b2bcbaf: Status 404 returned error can't find the container with id b438d8b094aba0dea4702c0ff583574112cc5bc8f6ccf34b9e58bead4b2bcbaf
Oct 31 01:24:34.661273 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.661237    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" event={"ID":"f4603662-92a5-474b-bff3-93a7fff42db2","Type":"ContainerStarted","Data":"c56a2dc88852d3d9943f3300d05770bac7ac1adf09f66f7769961c338fc4a49a"}
Oct 31 01:24:34.661497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.661293    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" event={"ID":"f4603662-92a5-474b-bff3-93a7fff42db2","Type":"ContainerStarted","Data":"bd3eb8a36ff12e59271c9104eff2c2c5d66bdbd75561dc000dcdcc28a2c98f80"}
Oct 31 01:24:34.661497 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.661351    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:34.663040 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.663017    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" event={"ID":"ffa18ae1-2dfc-411a-8b10-ba446d948077","Type":"ContainerStarted","Data":"cc022dfdbf76ac19bb4c63fb7027dec078df30b71306bd6ff304a88503480e89"}
Oct 31 01:24:34.663178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.663062    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" event={"ID":"ffa18ae1-2dfc-411a-8b10-ba446d948077","Type":"ContainerStarted","Data":"b438d8b094aba0dea4702c0ff583574112cc5bc8f6ccf34b9e58bead4b2bcbaf"}
Oct 31 01:24:34.663178 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.663089    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:34.683788 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.683740    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" podStartSLOduration=1.68372675 podStartE2EDuration="1.68372675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:24:34.6811442 +0000 UTC m=+1671.668256177" watchObservedRunningTime="2026-10-31 01:24:34.68372675 +0000 UTC m=+1671.670838775"
Oct 31 01:24:34.700143 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:34.700101    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" podStartSLOduration=1.7000874179999999 podStartE2EDuration="1.700087418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:24:34.697898834 +0000 UTC m=+1671.685010791" watchObservedRunningTime="2026-10-31 01:24:34.700087418 +0000 UTC m=+1671.687199396"
Oct 31 01:24:45.670951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:45.670921    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:24:45.673286 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:24:45.671227    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:24:49.442000 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:24:49.441960    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:25:19.450578 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:25:19.450491    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:25:19.913593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.913550    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"]
Oct 31 01:25:19.917714 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.917687    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:19.923823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.923784    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:25:19.923823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.923812    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:25:19.924054 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.924039    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:25:19.924225 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.924202    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:25:19.926722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:19.926700    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"]
Oct 31 01:25:20.029192 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.029150    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2s8b\" (UniqueName: \"kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.029388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.029213    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.029388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.029264    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.130038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.130002    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s8b\" (UniqueName: \"kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.130251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.130064    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.130251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.130095    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.130876 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.130848    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.132635 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.132593    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.140772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.140733    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2s8b\" (UniqueName: \"kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b\") pod \"maas-api-revoke-sub-7648cda3-l4f6c\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.233857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.233760    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:20.518204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.518173    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"]
Oct 31 01:25:20.520480 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:25:20.520445    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018b278a_01ce_4bbd_90ca_ee94799712fb.slice/crio-aacc77c6ebf801a4921a23ca83a4083206112119139dc073b3770075af807e1f WatchSource:0}: Error finding container aacc77c6ebf801a4921a23ca83a4083206112119139dc073b3770075af807e1f: Status 404 returned error can't find the container with id aacc77c6ebf801a4921a23ca83a4083206112119139dc073b3770075af807e1f
Oct 31 01:25:20.930021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.929907    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c" event={"ID":"018b278a-01ce-4bbd-90ca-ee94799712fb","Type":"ContainerStarted","Data":"94c2ed3afcb1a027e020f662dfe366765a59b5dcd007977ec09dc6c77883bc62"}
Oct 31 01:25:20.930021 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.929982    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c" event={"ID":"018b278a-01ce-4bbd-90ca-ee94799712fb","Type":"ContainerStarted","Data":"aacc77c6ebf801a4921a23ca83a4083206112119139dc073b3770075af807e1f"}
Oct 31 01:25:20.961924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:20.961864    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c" podStartSLOduration=1.961846663 podStartE2EDuration="1.961846663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:25:20.959829542 +0000 UTC m=+1717.946941519" watchObservedRunningTime="2026-10-31 01:25:20.961846663 +0000 UTC m=+1717.948958633"
Oct 31 01:25:21.936831 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:21.936793    8029 generic.go:356] "Generic (PLEG): container finished" podID="018b278a-01ce-4bbd-90ca-ee94799712fb" containerID="94c2ed3afcb1a027e020f662dfe366765a59b5dcd007977ec09dc6c77883bc62" exitCode=0
Oct 31 01:25:21.937251 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:21.936876    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c" event={"ID":"018b278a-01ce-4bbd-90ca-ee94799712fb","Type":"ContainerDied","Data":"94c2ed3afcb1a027e020f662dfe366765a59b5dcd007977ec09dc6c77883bc62"}
Oct 31 01:25:23.089658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.089592    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:23.264087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.263986    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b\" (UniqueName: \"kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b\") pod \"018b278a-01ce-4bbd-90ca-ee94799712fb\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") "
Oct 31 01:25:23.264087 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.264048    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca\") pod \"018b278a-01ce-4bbd-90ca-ee94799712fb\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") "
Oct 31 01:25:23.264311 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.264098    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca\") pod \"018b278a-01ce-4bbd-90ca-ee94799712fb\" (UID: \"018b278a-01ce-4bbd-90ca-ee94799712fb\") "
Oct 31 01:25:23.264688 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.264662    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca" pod "018b278a-01ce-4bbd-90ca-ee94799712fb" (UID: "018b278a-01ce-4bbd-90ca-ee94799712fb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:25:23.266469 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.266446    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b" pod "018b278a-01ce-4bbd-90ca-ee94799712fb" (UID: "018b278a-01ce-4bbd-90ca-ee94799712fb"). InnerVolumeSpecName "kube-api-access-r2s8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:25:23.266565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.266497    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca" pod "018b278a-01ce-4bbd-90ca-ee94799712fb" (UID: "018b278a-01ce-4bbd-90ca-ee94799712fb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:25:23.365354 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.365311    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:23.365354 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.365345    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2s8b\" (UniqueName: \"kubernetes.io/projected/018b278a-01ce-4bbd-90ca-ee94799712fb-kube-api-access-r2s8b\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:23.365354 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.365357    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/018b278a-01ce-4bbd-90ca-ee94799712fb-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:23.948490 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.948459    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"
Oct 31 01:25:23.948708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.948452    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c" event={"ID":"018b278a-01ce-4bbd-90ca-ee94799712fb","Type":"ContainerDied","Data":"aacc77c6ebf801a4921a23ca83a4083206112119139dc073b3770075af807e1f"}
Oct 31 01:25:23.948708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:23.948589    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aacc77c6ebf801a4921a23ca83a4083206112119139dc073b3770075af807e1f"
Oct 31 01:25:30.815835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.815797    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"]
Oct 31 01:25:30.816409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.816311    8029 cpu_manager.go:414] "containerMap: removing container" podUID="018b278a-01ce-4bbd-90ca-ee94799712fb" containerName="revoke-keys"
Oct 31 01:25:30.816409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.816326    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="018b278a-01ce-4bbd-90ca-ee94799712fb" containerName="revoke-keys"
Oct 31 01:25:30.816493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.816412    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="018b278a-01ce-4bbd-90ca-ee94799712fb" containerName="revoke-keys"
Oct 31 01:25:30.819727 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.819705    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.825197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.825165    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:25:30.826088 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.826065    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:25:30.828095 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.826417    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:25:30.828095 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.826581    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:25:30.829233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.829206    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"]
Oct 31 01:25:30.844925 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.844885    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtndl\" (UniqueName: \"kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.845125 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.845099    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.845256 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.845211    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.946612 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.946563    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.946810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.946671    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.946810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.946788    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtndl\" (UniqueName: \"kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.947306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.947281    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.949321 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.949293    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:30.955074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:30.955041    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtndl\" (UniqueName: \"kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl\") pod \"maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:31.138904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:31.138857    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:31.420220 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:31.420186    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"]
Oct 31 01:25:31.422912 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:25:31.422871    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27b4d7a_cef2_4a3e_9254_15003c2adf96.slice/crio-2b1f9efb193aa1b8f47c96f068c07f5e0dfed17c1db0007b81bf114f9171f768 WatchSource:0}: Error finding container 2b1f9efb193aa1b8f47c96f068c07f5e0dfed17c1db0007b81bf114f9171f768: Status 404 returned error can't find the container with id 2b1f9efb193aa1b8f47c96f068c07f5e0dfed17c1db0007b81bf114f9171f768
Oct 31 01:25:31.995924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:31.995804    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv" event={"ID":"a27b4d7a-cef2-4a3e-9254-15003c2adf96","Type":"ContainerStarted","Data":"3593da94beaac24407e533bee917df597eb8f966015518c968ab6f935b8fa5a1"}
Oct 31 01:25:31.995924 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:31.995884    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv" event={"ID":"a27b4d7a-cef2-4a3e-9254-15003c2adf96","Type":"ContainerStarted","Data":"2b1f9efb193aa1b8f47c96f068c07f5e0dfed17c1db0007b81bf114f9171f768"}
Oct 31 01:25:32.016034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:32.015969    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv" podStartSLOduration=2.015952925 podStartE2EDuration="2.015952925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:25:32.013729596 +0000 UTC m=+1729.000841571" watchObservedRunningTime="2026-10-31 01:25:32.015952925 +0000 UTC m=+1729.003064898"
Oct 31 01:25:33.002970 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:33.002925    8029 generic.go:356] "Generic (PLEG): container finished" podID="a27b4d7a-cef2-4a3e-9254-15003c2adf96" containerID="3593da94beaac24407e533bee917df597eb8f966015518c968ab6f935b8fa5a1" exitCode=0
Oct 31 01:25:33.003400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:33.003013    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv" event={"ID":"a27b4d7a-cef2-4a3e-9254-15003c2adf96","Type":"ContainerDied","Data":"3593da94beaac24407e533bee917df597eb8f966015518c968ab6f935b8fa5a1"}
Oct 31 01:25:34.165639 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.165579    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:34.182892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.182861    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca\") pod \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") "
Oct 31 01:25:34.183546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.182938    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl\" (UniqueName: \"kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl\") pod \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") "
Oct 31 01:25:34.183546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.183042    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca\") pod \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\" (UID: \"a27b4d7a-cef2-4a3e-9254-15003c2adf96\") "
Oct 31 01:25:34.183546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.183379    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca" pod "a27b4d7a-cef2-4a3e-9254-15003c2adf96" (UID: "a27b4d7a-cef2-4a3e-9254-15003c2adf96"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:25:34.186229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.186195    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca" pod "a27b4d7a-cef2-4a3e-9254-15003c2adf96" (UID: "a27b4d7a-cef2-4a3e-9254-15003c2adf96"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:25:34.186550 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.186522    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl" pod "a27b4d7a-cef2-4a3e-9254-15003c2adf96" (UID: "a27b4d7a-cef2-4a3e-9254-15003c2adf96"). InnerVolumeSpecName "kube-api-access-vtndl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:25:34.284938 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.284842    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:34.284938 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.284878    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a27b4d7a-cef2-4a3e-9254-15003c2adf96-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:34.284938 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:34.284892    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtndl\" (UniqueName: \"kubernetes.io/projected/a27b4d7a-cef2-4a3e-9254-15003c2adf96-kube-api-access-vtndl\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:25:35.015513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:35.015478    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"
Oct 31 01:25:35.015513 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:35.015501    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv" event={"ID":"a27b4d7a-cef2-4a3e-9254-15003c2adf96","Type":"ContainerDied","Data":"2b1f9efb193aa1b8f47c96f068c07f5e0dfed17c1db0007b81bf114f9171f768"}
Oct 31 01:25:35.015772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:35.015553    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b1f9efb193aa1b8f47c96f068c07f5e0dfed17c1db0007b81bf114f9171f768"
Oct 31 01:25:40.934526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:40.934480    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"]
Oct 31 01:25:40.947573 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:40.947535    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"]
Oct 31 01:25:42.159806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:42.159765    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"]
Oct 31 01:25:42.162751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:42.162712    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-b93c48-z2zqv"]
Oct 31 01:25:43.183934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:43.183900    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a27b4d7a-cef2-4a3e-9254-15003c2adf96" path="/var/lib/kubelet/pods/a27b4d7a-cef2-4a3e-9254-15003c2adf96/volumes"
Oct 31 01:25:45.935130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:45.935045    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerName="payload-processing" containerID="cri-o://cc022dfdbf76ac19bb4c63fb7027dec078df30b71306bd6ff304a88503480e89" gracePeriod=25
Oct 31 01:25:45.948732 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:45.948662    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" containerName="payload-pre-processing" containerID="cri-o://c56a2dc88852d3d9943f3300d05770bac7ac1adf09f66f7769961c338fc4a49a" gracePeriod=25
Oct 31 01:25:49.443549 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:25:49.443505    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:25:55.670299 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:55.670256    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:25:55.670759 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:25:55.670332    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:26:05.669399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:05.669353    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:26:05.670070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:05.669353    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 01:26:06.202745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.202700    8029 generic.go:356] "Generic (PLEG): container finished" podID="f4603662-92a5-474b-bff3-93a7fff42db2" containerID="c56a2dc88852d3d9943f3300d05770bac7ac1adf09f66f7769961c338fc4a49a" exitCode=0
Oct 31 01:26:06.202945 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.202766    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" event={"ID":"f4603662-92a5-474b-bff3-93a7fff42db2","Type":"ContainerDied","Data":"c56a2dc88852d3d9943f3300d05770bac7ac1adf09f66f7769961c338fc4a49a"}
Oct 31 01:26:06.204469 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.204443    8029 generic.go:356] "Generic (PLEG): container finished" podID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerID="cc022dfdbf76ac19bb4c63fb7027dec078df30b71306bd6ff304a88503480e89" exitCode=0
Oct 31 01:26:06.204615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.204491    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" event={"ID":"ffa18ae1-2dfc-411a-8b10-ba446d948077","Type":"ContainerDied","Data":"cc022dfdbf76ac19bb4c63fb7027dec078df30b71306bd6ff304a88503480e89"}
Oct 31 01:26:06.204615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.204535    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c" event={"ID":"ffa18ae1-2dfc-411a-8b10-ba446d948077","Type":"ContainerDied","Data":"b438d8b094aba0dea4702c0ff583574112cc5bc8f6ccf34b9e58bead4b2bcbaf"}
Oct 31 01:26:06.204615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.204559    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b438d8b094aba0dea4702c0ff583574112cc5bc8f6ccf34b9e58bead4b2bcbaf"
Oct 31 01:26:06.214631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.214583    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:26:06.231900 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.231868    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx\" (UniqueName: \"kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx\") pod \"ffa18ae1-2dfc-411a-8b10-ba446d948077\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") "
Oct 31 01:26:06.232094 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.232073    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume\") pod \"ffa18ae1-2dfc-411a-8b10-ba446d948077\" (UID: \"ffa18ae1-2dfc-411a-8b10-ba446d948077\") "
Oct 31 01:26:06.232683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.232579    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume" pod "ffa18ae1-2dfc-411a-8b10-ba446d948077" (UID: "ffa18ae1-2dfc-411a-8b10-ba446d948077"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:26:06.236687 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.235232    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx" pod "ffa18ae1-2dfc-411a-8b10-ba446d948077" (UID: "ffa18ae1-2dfc-411a-8b10-ba446d948077"). InnerVolumeSpecName "kube-api-access-wh6dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:26:06.247875 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.247840    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:26:06.333151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.333106    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume\") pod \"f4603662-92a5-474b-bff3-93a7fff42db2\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") "
Oct 31 01:26:06.333375 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.333218    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns\" (UniqueName: \"kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns\") pod \"f4603662-92a5-474b-bff3-93a7fff42db2\" (UID: \"f4603662-92a5-474b-bff3-93a7fff42db2\") "
Oct 31 01:26:06.333459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.333439    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa18ae1-2dfc-411a-8b10-ba446d948077-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:06.333459 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.333450    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh6dx\" (UniqueName: \"kubernetes.io/projected/ffa18ae1-2dfc-411a-8b10-ba446d948077-kube-api-access-wh6dx\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:06.333573 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.333520    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume" pod "f4603662-92a5-474b-bff3-93a7fff42db2" (UID: "f4603662-92a5-474b-bff3-93a7fff42db2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:26:06.335647 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.335619    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns" pod "f4603662-92a5-474b-bff3-93a7fff42db2" (UID: "f4603662-92a5-474b-bff3-93a7fff42db2"). InnerVolumeSpecName "kube-api-access-k64ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:26:06.434386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.434340    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4603662-92a5-474b-bff3-93a7fff42db2-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:06.434386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:06.434375    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k64ns\" (UniqueName: \"kubernetes.io/projected/f4603662-92a5-474b-bff3-93a7fff42db2-kube-api-access-k64ns\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:07.211706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.211584    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"
Oct 31 01:26:07.211706 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.211650    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz" event={"ID":"f4603662-92a5-474b-bff3-93a7fff42db2","Type":"ContainerDied","Data":"bd3eb8a36ff12e59271c9104eff2c2c5d66bdbd75561dc000dcdcc28a2c98f80"}
Oct 31 01:26:07.212228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.211713    8029 scope.go:122] "RemoveContainer" containerID="c56a2dc88852d3d9943f3300d05770bac7ac1adf09f66f7769961c338fc4a49a"
Oct 31 01:26:07.212228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.211846    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"
Oct 31 01:26:07.230342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.230305    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"]
Oct 31 01:26:07.236868 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.236833    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b93c48-5c578b554jrg2c"]
Oct 31 01:26:07.247336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.247300    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"]
Oct 31 01:26:07.252888 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:07.252856    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b93c48-5bbdfn8xqz"]
Oct 31 01:26:09.182790 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:09.182760    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" path="/var/lib/kubelet/pods/f4603662-92a5-474b-bff3-93a7fff42db2/volumes"
Oct 31 01:26:09.183208 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:09.183165    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" path="/var/lib/kubelet/pods/ffa18ae1-2dfc-411a-8b10-ba446d948077/volumes"
Oct 31 01:26:19.444092 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:26:19.444042    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:26:26.732558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.732520    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"]
Oct 31 01:26:26.733030 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733015    8029 cpu_manager.go:414] "containerMap: removing container" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" containerName="payload-pre-processing"
Oct 31 01:26:26.733076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733032    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" containerName="payload-pre-processing"
Oct 31 01:26:26.733076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733060    8029 cpu_manager.go:414] "containerMap: removing container" podUID="a27b4d7a-cef2-4a3e-9254-15003c2adf96" containerName="revoke-keys"
Oct 31 01:26:26.733076 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733066    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a27b4d7a-cef2-4a3e-9254-15003c2adf96" containerName="revoke-keys"
Oct 31 01:26:26.733171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733077    8029 cpu_manager.go:414] "containerMap: removing container" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerName="payload-processing"
Oct 31 01:26:26.733171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733082    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerName="payload-processing"
Oct 31 01:26:26.733171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733156    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a27b4d7a-cef2-4a3e-9254-15003c2adf96" containerName="revoke-keys"
Oct 31 01:26:26.733171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733166    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffa18ae1-2dfc-411a-8b10-ba446d948077" containerName="payload-processing"
Oct 31 01:26:26.733361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.733176    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4603662-92a5-474b-bff3-93a7fff42db2" containerName="payload-pre-processing"
Oct 31 01:26:26.738124 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.738095    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.742968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.742936    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:26:26.742968 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.742978    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:26:26.745307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.745268    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"]
Oct 31 01:26:26.815593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.815554    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:26:26.815818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.815696    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:26:26.893592 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.893556    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5qjm\" (UniqueName: \"kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.893804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.893708    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.893804 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.893759    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.994721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.994578    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qjm\" (UniqueName: \"kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.994721 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.994698    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.994977 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.994741    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.995421 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.995394    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:26.997448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:26.997425    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:27.002437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:27.002410    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5qjm\" (UniqueName: \"kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm\") pod \"maas-api-revoke-sub-452b6d7b-hfc6h\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:27.055406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:27.055364    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:27.334320 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:27.334291    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"]
Oct 31 01:26:27.337104 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:26:27.337074    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b35cc54_c378_4f12_a45c_5beaf665c60d.slice/crio-6b190c2ae12650a542f66c2bc527988262a421ad8cd756f36af71b5f9f53f3e3 WatchSource:0}: Error finding container 6b190c2ae12650a542f66c2bc527988262a421ad8cd756f36af71b5f9f53f3e3: Status 404 returned error can't find the container with id 6b190c2ae12650a542f66c2bc527988262a421ad8cd756f36af71b5f9f53f3e3
Oct 31 01:26:28.345670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:28.345525    8029 generic.go:356] "Generic (PLEG): container finished" podID="7b35cc54-c378-4f12-a45c-5beaf665c60d" containerID="fd4aaf8d68804b878831415d98c859c72f13419ed65f71aa29fb4d357cb19359" exitCode=0
Oct 31 01:26:28.345670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:28.345590    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h" event={"ID":"7b35cc54-c378-4f12-a45c-5beaf665c60d","Type":"ContainerDied","Data":"fd4aaf8d68804b878831415d98c859c72f13419ed65f71aa29fb4d357cb19359"}
Oct 31 01:26:28.345670 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:28.345665    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h" event={"ID":"7b35cc54-c378-4f12-a45c-5beaf665c60d","Type":"ContainerStarted","Data":"6b190c2ae12650a542f66c2bc527988262a421ad8cd756f36af71b5f9f53f3e3"}
Oct 31 01:26:29.749041 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.748973    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:29.861850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.861049    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm\" (UniqueName: \"kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm\") pod \"7b35cc54-c378-4f12-a45c-5beaf665c60d\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") "
Oct 31 01:26:29.861850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.861132    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca\") pod \"7b35cc54-c378-4f12-a45c-5beaf665c60d\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") "
Oct 31 01:26:29.861850 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.861197    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca\") pod \"7b35cc54-c378-4f12-a45c-5beaf665c60d\" (UID: \"7b35cc54-c378-4f12-a45c-5beaf665c60d\") "
Oct 31 01:26:29.863237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.863206    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca" pod "7b35cc54-c378-4f12-a45c-5beaf665c60d" (UID: "7b35cc54-c378-4f12-a45c-5beaf665c60d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:26:29.872070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.871741    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca" pod "7b35cc54-c378-4f12-a45c-5beaf665c60d" (UID: "7b35cc54-c378-4f12-a45c-5beaf665c60d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:26:29.872070 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.871975    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm" pod "7b35cc54-c378-4f12-a45c-5beaf665c60d" (UID: "7b35cc54-c378-4f12-a45c-5beaf665c60d"). InnerVolumeSpecName "kube-api-access-h5qjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:26:29.962210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.962102    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5qjm\" (UniqueName: \"kubernetes.io/projected/7b35cc54-c378-4f12-a45c-5beaf665c60d-kube-api-access-h5qjm\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:29.962210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.962133    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:29.962210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:29.962144    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7b35cc54-c378-4f12-a45c-5beaf665c60d-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:30.365084 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:30.365046    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"
Oct 31 01:26:30.365084 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:30.365078    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h" event={"ID":"7b35cc54-c378-4f12-a45c-5beaf665c60d","Type":"ContainerDied","Data":"6b190c2ae12650a542f66c2bc527988262a421ad8cd756f36af71b5f9f53f3e3"}
Oct 31 01:26:30.365362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:30.365124    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b190c2ae12650a542f66c2bc527988262a421ad8cd756f36af71b5f9f53f3e3"
Oct 31 01:26:41.021913 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.021865    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"]
Oct 31 01:26:41.023678 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.023647    8029 cpu_manager.go:414] "containerMap: removing container" podUID="7b35cc54-c378-4f12-a45c-5beaf665c60d" containerName="revoke-keys"
Oct 31 01:26:41.023886 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.023869    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b35cc54-c378-4f12-a45c-5beaf665c60d" containerName="revoke-keys"
Oct 31 01:26:41.024197 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.024180    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b35cc54-c378-4f12-a45c-5beaf665c60d" containerName="revoke-keys"
Oct 31 01:26:41.034697 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.033294    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.052341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.052224    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:26:41.057506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.053291    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:26:41.057506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.054064    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:26:41.060387 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.057913    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:26:41.061733 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.061287    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"]
Oct 31 01:26:41.085336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.085294    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.104859 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.104814    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.105398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.105365    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99ncf\" (UniqueName: \"kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.228628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.228247    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99ncf\" (UniqueName: \"kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.228628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.228317    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.228628 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.228381    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.251981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.243537    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.251981 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.250971    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.256659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.254318    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99ncf\" (UniqueName: \"kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf\") pod \"maas-api-revoke-sub-3e31a506-4ndrk\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.374368 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.373947    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:41.826707 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:41.818973    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"]
Oct 31 01:26:42.461937 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:42.461847    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk" event={"ID":"a9774d05-55fd-4a32-8509-eb125fc6f8ec","Type":"ContainerStarted","Data":"ad37cde945cf651847a7669ed3ed8a05594baa064ca477080a93028911c09fe6"}
Oct 31 01:26:42.461937 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:42.461923    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk" event={"ID":"a9774d05-55fd-4a32-8509-eb125fc6f8ec","Type":"ContainerStarted","Data":"71016700fc4529281206118573b8547c0bc6862fb13ea70207fddfa127f5c7de"}
Oct 31 01:26:43.468971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:43.468933    8029 generic.go:356] "Generic (PLEG): container finished" podID="a9774d05-55fd-4a32-8509-eb125fc6f8ec" containerID="ad37cde945cf651847a7669ed3ed8a05594baa064ca477080a93028911c09fe6" exitCode=0
Oct 31 01:26:43.469498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:43.469038    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk" event={"ID":"a9774d05-55fd-4a32-8509-eb125fc6f8ec","Type":"ContainerDied","Data":"ad37cde945cf651847a7669ed3ed8a05594baa064ca477080a93028911c09fe6"}
Oct 31 01:26:43.493583 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:43.493540    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:26:43.495200 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:43.495174    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:26:44.970553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:44.970152    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:45.029306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.029253    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca\") pod \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") "
Oct 31 01:26:45.030119 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.030093    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca\") pod \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") "
Oct 31 01:26:45.030840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.030818    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf\" (UniqueName: \"kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf\") pod \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\" (UID: \"a9774d05-55fd-4a32-8509-eb125fc6f8ec\") "
Oct 31 01:26:45.043575 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.030015    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca" pod "a9774d05-55fd-4a32-8509-eb125fc6f8ec" (UID: "a9774d05-55fd-4a32-8509-eb125fc6f8ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:26:45.053842 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.053800    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca" pod "a9774d05-55fd-4a32-8509-eb125fc6f8ec" (UID: "a9774d05-55fd-4a32-8509-eb125fc6f8ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:26:45.056562 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.056499    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf" pod "a9774d05-55fd-4a32-8509-eb125fc6f8ec" (UID: "a9774d05-55fd-4a32-8509-eb125fc6f8ec"). InnerVolumeSpecName "kube-api-access-99ncf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:26:45.147696 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.147378    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99ncf\" (UniqueName: \"kubernetes.io/projected/a9774d05-55fd-4a32-8509-eb125fc6f8ec-kube-api-access-99ncf\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:45.147696 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.147437    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:45.147696 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.147455    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a9774d05-55fd-4a32-8509-eb125fc6f8ec-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:26:45.564518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.564408    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk" event={"ID":"a9774d05-55fd-4a32-8509-eb125fc6f8ec","Type":"ContainerDied","Data":"71016700fc4529281206118573b8547c0bc6862fb13ea70207fddfa127f5c7de"}
Oct 31 01:26:45.564518 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.564485    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71016700fc4529281206118573b8547c0bc6862fb13ea70207fddfa127f5c7de"
Oct 31 01:26:45.564873 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.564563    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"
Oct 31 01:26:45.875297 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:45.875259    8029 scope.go:122] "RemoveContainer" containerID="375e979ac4a7b6c4e67b5f8666b937133765bb8b1e8a00383a28db6b32695be6"
Oct 31 01:26:47.027654 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:47.027592    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"]
Oct 31 01:26:47.029745 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:47.029707    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-991e3f8a-ttqmm"]
Oct 31 01:26:47.182062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:26:47.182025    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99a22103-042b-47c1-93a7-6a76685eede3" path="/var/lib/kubelet/pods/99a22103-042b-47c1-93a7-6a76685eede3/volumes"
Oct 31 01:26:49.445272 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:26:49.445179    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:27:01.563815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.563768    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"]
Oct 31 01:27:01.564281 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.564266    8029 cpu_manager.go:414] "containerMap: removing container" podUID="a9774d05-55fd-4a32-8509-eb125fc6f8ec" containerName="revoke-keys"
Oct 31 01:27:01.564325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.564282    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9774d05-55fd-4a32-8509-eb125fc6f8ec" containerName="revoke-keys"
Oct 31 01:27:01.564379 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.564369    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9774d05-55fd-4a32-8509-eb125fc6f8ec" containerName="revoke-keys"
Oct 31 01:27:01.567631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.567596    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.576259 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.576226    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:27:01.576426 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.576358    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:27:01.577160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.577138    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:27:01.577160 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.577152    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:27:01.578803 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.578775    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"]
Oct 31 01:27:01.695470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.695433    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.695743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.695570    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.695743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.695681    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qp2g\" (UniqueName: \"kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.796323 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.796276    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.796531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.796392    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.796531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.796468    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qp2g\" (UniqueName: \"kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.797102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.797077    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.798866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.798847    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.805514 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.805488    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qp2g\" (UniqueName: \"kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g\") pod \"maas-api-revoke-sub-05301312-z5tcs\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:01.882722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:01.882689    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:02.164018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:02.163987    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"]
Oct 31 01:27:02.166705 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:27:02.166666    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889079e9_4c68_466e_af06_e9f7073e9dea.slice/crio-f3830c6bcc41407b3f802f086ef43e64b41778a63c42d4ad612dae7d715716b3 WatchSource:0}: Error finding container f3830c6bcc41407b3f802f086ef43e64b41778a63c42d4ad612dae7d715716b3: Status 404 returned error can't find the container with id f3830c6bcc41407b3f802f086ef43e64b41778a63c42d4ad612dae7d715716b3
Oct 31 01:27:02.697153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:02.697110    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs" event={"ID":"889079e9-4c68-466e-af06-e9f7073e9dea","Type":"ContainerStarted","Data":"c9bc762b82d359f719e265ec0e9b737bc7ecc9294472a90eca2c0c91f3318498"}
Oct 31 01:27:02.697619 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:02.697180    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs" event={"ID":"889079e9-4c68-466e-af06-e9f7073e9dea","Type":"ContainerStarted","Data":"f3830c6bcc41407b3f802f086ef43e64b41778a63c42d4ad612dae7d715716b3"}
Oct 31 01:27:02.715673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:02.715589    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs" podStartSLOduration=1.715573571 podStartE2EDuration="1.715573571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:27:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:27:02.71374119 +0000 UTC m=+1819.700853166" watchObservedRunningTime="2026-10-31 01:27:02.715573571 +0000 UTC m=+1819.702685547"
Oct 31 01:27:03.704189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:03.704147    8029 generic.go:356] "Generic (PLEG): container finished" podID="889079e9-4c68-466e-af06-e9f7073e9dea" containerID="c9bc762b82d359f719e265ec0e9b737bc7ecc9294472a90eca2c0c91f3318498" exitCode=0
Oct 31 01:27:03.704641 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:03.704229    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs" event={"ID":"889079e9-4c68-466e-af06-e9f7073e9dea","Type":"ContainerDied","Data":"c9bc762b82d359f719e265ec0e9b737bc7ecc9294472a90eca2c0c91f3318498"}
Oct 31 01:27:04.866155 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:04.866124    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:05.030038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.029930    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g\" (UniqueName: \"kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g\") pod \"889079e9-4c68-466e-af06-e9f7073e9dea\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") "
Oct 31 01:27:05.030038 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.030001    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca\") pod \"889079e9-4c68-466e-af06-e9f7073e9dea\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") "
Oct 31 01:27:05.030286 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.030097    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca\") pod \"889079e9-4c68-466e-af06-e9f7073e9dea\" (UID: \"889079e9-4c68-466e-af06-e9f7073e9dea\") "
Oct 31 01:27:05.030453 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.030427    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca" pod "889079e9-4c68-466e-af06-e9f7073e9dea" (UID: "889079e9-4c68-466e-af06-e9f7073e9dea"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:05.032397 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.032363    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca" pod "889079e9-4c68-466e-af06-e9f7073e9dea" (UID: "889079e9-4c68-466e-af06-e9f7073e9dea"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:27:05.032524 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.032429    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g" pod "889079e9-4c68-466e-af06-e9f7073e9dea" (UID: "889079e9-4c68-466e-af06-e9f7073e9dea"). InnerVolumeSpecName "kube-api-access-4qp2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:05.131448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.131400    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:05.131448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.131435    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qp2g\" (UniqueName: \"kubernetes.io/projected/889079e9-4c68-466e-af06-e9f7073e9dea-kube-api-access-4qp2g\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:05.131448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.131446    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/889079e9-4c68-466e-af06-e9f7073e9dea-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:05.717012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.716977    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"
Oct 31 01:27:05.717202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.716976    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs" event={"ID":"889079e9-4c68-466e-af06-e9f7073e9dea","Type":"ContainerDied","Data":"f3830c6bcc41407b3f802f086ef43e64b41778a63c42d4ad612dae7d715716b3"}
Oct 31 01:27:05.717202 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:05.717126    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3830c6bcc41407b3f802f086ef43e64b41778a63c42d4ad612dae7d715716b3"
Oct 31 01:27:18.032901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:18.032862    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"]
Oct 31 01:27:18.034557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:18.034529    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-652nq"]
Oct 31 01:27:19.182063 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:19.182026    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98437962-3278-4948-8b48-c986dce13b9a" path="/var/lib/kubelet/pods/98437962-3278-4948-8b48-c986dce13b9a/volumes"
Oct 31 01:27:19.444144 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:27:19.444053    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:27:21.667425 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.667380    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"]
Oct 31 01:27:21.668042 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.668013    8029 cpu_manager.go:414] "containerMap: removing container" podUID="889079e9-4c68-466e-af06-e9f7073e9dea" containerName="revoke-keys"
Oct 31 01:27:21.668108 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.668045    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="889079e9-4c68-466e-af06-e9f7073e9dea" containerName="revoke-keys"
Oct 31 01:27:21.668159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.668150    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="889079e9-4c68-466e-af06-e9f7073e9dea" containerName="revoke-keys"
Oct 31 01:27:21.673044 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.673019    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.692257 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.692231    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b68f43ca-openshift-default-dockercfg-vwl5x\""
Oct 31 01:27:21.702778 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.702752    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.702906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.702802    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.702906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.702823    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.702991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.702960    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.702991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.702984    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.703074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.703005    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x7f2\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.703074 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.703024    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.703163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.703111    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.703224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.703212    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.764777 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.764741    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"]
Oct 31 01:27:21.804187 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804140    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804230    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804260    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804303    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804408 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804326    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804500    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x7f2\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804540    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804686 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804582    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804853 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804735    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804913 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804853    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.804979 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.804925    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.805114 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.805093    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.805255 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.805233    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.805339 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.805316    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.806996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.806970    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.807579 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.807551    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.828549 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.828517    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.828912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.828887    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x7f2\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2\") pod \"e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") " pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:21.987199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:21.987093    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:22.277773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.277652    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"]
Oct 31 01:27:22.280559 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:27:22.280521    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba5bc2b_8070_4321_a9ac_79b28e844103.slice/crio-2daa9dcbc85694dcf8da4afec3aca9b5fabb31c5d1fc3001da774b3bafe8cb76 WatchSource:0}: Error finding container 2daa9dcbc85694dcf8da4afec3aca9b5fabb31c5d1fc3001da774b3bafe8cb76: Status 404 returned error can't find the container with id 2daa9dcbc85694dcf8da4afec3aca9b5fabb31c5d1fc3001da774b3bafe8cb76
Oct 31 01:27:22.283577 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.283524    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:22.283748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.283658    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:22.283748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.283704    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:22.809909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.809864    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" event={"ID":"dba5bc2b-8070-4321-a9ac-79b28e844103","Type":"ContainerStarted","Data":"a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"}
Oct 31 01:27:22.810388 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.809933    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" event={"ID":"dba5bc2b-8070-4321-a9ac-79b28e844103","Type":"ContainerStarted","Data":"2daa9dcbc85694dcf8da4afec3aca9b5fabb31c5d1fc3001da774b3bafe8cb76"}
Oct 31 01:27:22.832353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.832292    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podStartSLOduration=1.832276651 podStartE2EDuration="1.832276651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:27:22.831026644 +0000 UTC m=+1839.818138649" watchObservedRunningTime="2026-10-31 01:27:22.832276651 +0000 UTC m=+1839.819388693"
Oct 31 01:27:22.987523 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:22.987485    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:23.993316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:23.993277    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:24.821090 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:24.821054    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:24.822188 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:24.822166    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:25.683527 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.683486    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"]
Oct 31 01:27:25.687693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.687674    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.711113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.711074    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b68f43ca-dockercfg-mwwnw\""
Oct 31 01:27:25.712423 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.712395    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"]
Oct 31 01:27:25.719644 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.717618    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b68f43ca\""
Oct 31 01:27:25.748353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.748306    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgq6\" (UniqueName: \"kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6\") pod \"payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.748353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.748353    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.750264 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.750230    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"]
Oct 31 01:27:25.755533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.755492    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:25.768309 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.768240    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"]
Oct 31 01:27:25.849181 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.849142    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") " pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:25.849409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.849325    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ltxr\" (UniqueName: \"kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr\") pod \"payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") " pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:25.849409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.849362    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgq6\" (UniqueName: \"kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6\") pod \"payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.849409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.849385    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.850175 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.850155    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.858743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.858715    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgq6\" (UniqueName: \"kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6\") pod \"payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:25.950761 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.950654    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") " pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:25.950955 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.950803    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ltxr\" (UniqueName: \"kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr\") pod \"payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") " pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:25.951506 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.951479    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume\") pod \"payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") " pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:25.964031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:25.963988    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ltxr\" (UniqueName: \"kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr\") pod \"payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") " pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:26.002805 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.002765    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:26.073520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.073475    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:26.291796 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.291763    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"]
Oct 31 01:27:26.293924 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:27:26.293894    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3fd78f_7888_4741_a9b2_b6d082f7fa67.slice/crio-d957fbf3fc6f2cc959d504188783e5649f3067643d0ea81784e1f31f58c07b51 WatchSource:0}: Error finding container d957fbf3fc6f2cc959d504188783e5649f3067643d0ea81784e1f31f58c07b51: Status 404 returned error can't find the container with id d957fbf3fc6f2cc959d504188783e5649f3067643d0ea81784e1f31f58c07b51
Oct 31 01:27:26.379350 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.379314    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"]
Oct 31 01:27:26.383657 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:27:26.383623    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aba2d04_fc4a_4df9_87cb_dbf818d45c3a.slice/crio-e0662719478f89ea703cb2d3ec23574a74a7571f45e5b409f6305b063ac0bc62 WatchSource:0}: Error finding container e0662719478f89ea703cb2d3ec23574a74a7571f45e5b409f6305b063ac0bc62: Status 404 returned error can't find the container with id e0662719478f89ea703cb2d3ec23574a74a7571f45e5b409f6305b063ac0bc62
Oct 31 01:27:26.836539 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.836125    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w" event={"ID":"1f3fd78f-7888-4741-a9b2-b6d082f7fa67","Type":"ContainerStarted","Data":"0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0"}
Oct 31 01:27:26.836539 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.836194    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w" event={"ID":"1f3fd78f-7888-4741-a9b2-b6d082f7fa67","Type":"ContainerStarted","Data":"d957fbf3fc6f2cc959d504188783e5649f3067643d0ea81784e1f31f58c07b51"}
Oct 31 01:27:26.836539 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.836447    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:26.842715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.842670    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs" event={"ID":"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a","Type":"ContainerStarted","Data":"257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08"}
Oct 31 01:27:26.842901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.842771    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs" event={"ID":"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a","Type":"ContainerStarted","Data":"e0662719478f89ea703cb2d3ec23574a74a7571f45e5b409f6305b063ac0bc62"}
Oct 31 01:27:26.842901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.842807    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:26.859891 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.859833    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w" podStartSLOduration=1.859817791 podStartE2EDuration="1.859817791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:27:26.857827362 +0000 UTC m=+1843.844939411" watchObservedRunningTime="2026-10-31 01:27:26.859817791 +0000 UTC m=+1843.846929845"
Oct 31 01:27:26.881544 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:26.881358    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs" podStartSLOduration=1.8813421460000002 podStartE2EDuration="1.881342146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:27:26.880924778 +0000 UTC m=+1843.868036756" watchObservedRunningTime="2026-10-31 01:27:26.881342146 +0000 UTC m=+1843.868454123"
Oct 31 01:27:37.850747 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:37.850710    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:38.855130 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.855100    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:38.938233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.938146    8029 generic.go:356] "Generic (PLEG): container finished" podID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerID="a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f" exitCode=0
Oct 31 01:27:38.938233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.938218    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" event={"ID":"dba5bc2b-8070-4321-a9ac-79b28e844103","Type":"ContainerDied","Data":"a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"}
Oct 31 01:27:38.938662 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.938645    8029 scope.go:122] "RemoveContainer" containerID="a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"
Oct 31 01:27:38.940175 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.940135    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:38.940273 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.940254    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:38.940327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:38.940312    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:39.821931 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:39.821878    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:39.947794 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:39.947760    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" event={"ID":"dba5bc2b-8070-4321-a9ac-79b28e844103","Type":"ContainerStarted","Data":"3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1"}
Oct 31 01:27:39.994627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:39.991736    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:40.260342 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.260302    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"]
Oct 31 01:27:40.264131 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.264113    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.268763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.268738    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:27:40.268905 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.268738    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs6l6\""
Oct 31 01:27:40.268953 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.268938    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:27:40.269204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.269187    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:27:40.274685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.274655    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"]
Oct 31 01:27:40.421761 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.421718    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6q6l\" (UniqueName: \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.421962 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.421775    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.421962 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.421891    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.522989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.522895    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6q6l\" (UniqueName: \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.522989 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.522956    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.523239 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.523032    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.523853 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.523745    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.525646 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:40.525623    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:40.531006 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:27:40.530985    8029 projected.go:196] Error preparing data for projected volume kube-api-access-f6q6l for pod odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp: failed to fetch token: serviceaccounts "maas-api-cleanup" not found
Oct 31 01:27:40.531083 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:27:40.531070    8029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l podName:e2b36dc5-9571-400f-8c6c-19abedb02662 nodeName:}" failed. No retries permitted until 2026-10-31 01:27:41.031053271 +0000 UTC m=+1858.018165225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f6q6l" (UniqueName: "kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l") pod "maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" (UID: "e2b36dc5-9571-400f-8c6c-19abedb02662") : failed to fetch token: serviceaccounts "maas-api-cleanup" not found
Oct 31 01:27:41.001785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.001746    8029 patch_prober.go:29] interesting pod/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 01:27:41.002372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.001809    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 01:27:41.130417 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.130368    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6q6l\" (UniqueName: \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:41.140390 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.140366    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6q6l\" (UniqueName: \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\") pod \"maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:41.208627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.206820    8029 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" secret="" err="secret \"maas-api-cleanup-dockercfg-zs6l6\" not found"
Oct 31 01:27:41.208627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.206914    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:41.503334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.503305    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"]
Oct 31 01:27:41.960229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.960191    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" event={"ID":"e2b36dc5-9571-400f-8c6c-19abedb02662","Type":"ContainerStarted","Data":"3fb7873a986be370f14252f09226fc6a1c127742730d7720b7b930702e063c91"}
Oct 31 01:27:41.960429 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.960250    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" event={"ID":"e2b36dc5-9571-400f-8c6c-19abedb02662","Type":"ContainerStarted","Data":"d3820024077474eb63440c7494833c161a83870078271e677c238262bf923edf"}
Oct 31 01:27:41.960507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.960485    8029 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" secret="" err="secret \"maas-api-cleanup-dockercfg-zs6l6\" not found"
Oct 31 01:27:41.992825 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.992783    8029 patch_prober.go:29] interesting pod/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 01:27:41.993125 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:41.993098    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 01:27:42.968382 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:42.968346    8029 generic.go:356] "Generic (PLEG): container finished" podID="e2b36dc5-9571-400f-8c6c-19abedb02662" containerID="3fb7873a986be370f14252f09226fc6a1c127742730d7720b7b930702e063c91" exitCode=0
Oct 31 01:27:42.968921 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:42.968434    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" event={"ID":"e2b36dc5-9571-400f-8c6c-19abedb02662","Type":"ContainerDied","Data":"3fb7873a986be370f14252f09226fc6a1c127742730d7720b7b930702e063c91"}
Oct 31 01:27:42.998766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:42.998722    8029 patch_prober.go:29] interesting pod/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 01:27:42.999028 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:42.998999    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 01:27:44.001830 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.001785    8029 patch_prober.go:29] interesting pod/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 01:27:44.002352 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.002160    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 01:27:44.210922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.210890    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:44.267162 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.267070    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca\") pod \"e2b36dc5-9571-400f-8c6c-19abedb02662\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") "
Oct 31 01:27:44.267332 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.267209    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca\") pod \"e2b36dc5-9571-400f-8c6c-19abedb02662\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") "
Oct 31 01:27:44.267332 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.267273    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\" (UniqueName: \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\") pod \"e2b36dc5-9571-400f-8c6c-19abedb02662\" (UID: \"e2b36dc5-9571-400f-8c6c-19abedb02662\") "
Oct 31 01:27:44.267631 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.267584    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca" pod "e2b36dc5-9571-400f-8c6c-19abedb02662" (UID: "e2b36dc5-9571-400f-8c6c-19abedb02662"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:44.268409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.268122    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:44.269723 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.269700    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca" pod "e2b36dc5-9571-400f-8c6c-19abedb02662" (UID: "e2b36dc5-9571-400f-8c6c-19abedb02662"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:27:44.270182 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.270162    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l" pod "e2b36dc5-9571-400f-8c6c-19abedb02662" (UID: "e2b36dc5-9571-400f-8c6c-19abedb02662"). InnerVolumeSpecName "kube-api-access-f6q6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:44.369329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.369278    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2b36dc5-9571-400f-8c6c-19abedb02662-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:44.369329 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.369315    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6q6l\" (UniqueName: \"kubernetes.io/projected/e2b36dc5-9571-400f-8c6c-19abedb02662-kube-api-access-f6q6l\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:44.982768 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.982723    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp" event={"ID":"e2b36dc5-9571-400f-8c6c-19abedb02662","Type":"ContainerDied","Data":"d3820024077474eb63440c7494833c161a83870078271e677c238262bf923edf"}
Oct 31 01:27:44.983000 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.982788    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3820024077474eb63440c7494833c161a83870078271e677c238262bf923edf"
Oct 31 01:27:44.983000 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.982794    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"
Oct 31 01:27:44.987620 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.987557    8029 patch_prober.go:29] interesting pod/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 01:27:44.987816 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:44.987670    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 01:27:45.949527 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:45.949485    8029 scope.go:122] "RemoveContainer" containerID="7621fecf6df443e92568dbbe69afc2314494b549562b240f78d7e0cf0343a4c8"
Oct 31 01:27:45.993163 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:45.993125    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:45.993666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:45.993642    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:45.994564 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:45.994547    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:27:49.443594 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:27:49.443551    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:27:50.432316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:50.432272    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"]
Oct 31 01:27:50.443910 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:50.443870    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"]
Oct 31 01:27:51.400169 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:51.400127    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"]
Oct 31 01:27:51.403627 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:51.403572    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b68f43ca-h5ffp"]
Oct 31 01:27:53.182366 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:53.182325    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2b36dc5-9571-400f-8c6c-19abedb02662" path="/var/lib/kubelet/pods/e2b36dc5-9571-400f-8c6c-19abedb02662/volumes"
Oct 31 01:27:55.433048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.432978    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs" podUID="0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" containerName="payload-processing" containerID="cri-o://257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08" gracePeriod=25
Oct 31 01:27:55.444630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.444557    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w" podUID="1f3fd78f-7888-4741-a9b2-b6d082f7fa67" containerName="payload-pre-processing" containerID="cri-o://0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0" gracePeriod=25
Oct 31 01:27:55.704890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.704857    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:55.736683 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.736645    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:55.798102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.798058    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr\" (UniqueName: \"kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr\") pod \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") "
Oct 31 01:27:55.798302 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.798166    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume\") pod \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\" (UID: \"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a\") "
Oct 31 01:27:55.798533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.798509    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume" pod "0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" (UID: "0aba2d04-fc4a-4df9-87cb-dbf818d45c3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:55.798763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.798741    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:55.800648 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.800624    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr" pod "0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" (UID: "0aba2d04-fc4a-4df9-87cb-dbf818d45c3a"). InnerVolumeSpecName "kube-api-access-4ltxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:55.899436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.899400    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6\" (UniqueName: \"kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6\") pod \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") "
Oct 31 01:27:55.899678 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.899463    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume\") pod \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\" (UID: \"1f3fd78f-7888-4741-a9b2-b6d082f7fa67\") "
Oct 31 01:27:55.899771 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.899754    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ltxr\" (UniqueName: \"kubernetes.io/projected/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a-kube-api-access-4ltxr\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:55.899877 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.899854    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume" pod "1f3fd78f-7888-4741-a9b2-b6d082f7fa67" (UID: "1f3fd78f-7888-4741-a9b2-b6d082f7fa67"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:27:55.901860 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:55.901838    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6" pod "1f3fd78f-7888-4741-a9b2-b6d082f7fa67" (UID: "1f3fd78f-7888-4741-a9b2-b6d082f7fa67"). InnerVolumeSpecName "kube-api-access-ttgq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:27:56.001215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.001125    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttgq6\" (UniqueName: \"kubernetes.io/projected/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-kube-api-access-ttgq6\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:56.001215 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.001158    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3fd78f-7888-4741-a9b2-b6d082f7fa67-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:27:56.055381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.055339    8029 generic.go:356] "Generic (PLEG): container finished" podID="1f3fd78f-7888-4741-a9b2-b6d082f7fa67" containerID="0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0" exitCode=0
Oct 31 01:27:56.055594 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.055404    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"
Oct 31 01:27:56.055594 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.055425    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w" event={"ID":"1f3fd78f-7888-4741-a9b2-b6d082f7fa67","Type":"ContainerDied","Data":"0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0"}
Oct 31 01:27:56.055594 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.055526    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w" event={"ID":"1f3fd78f-7888-4741-a9b2-b6d082f7fa67","Type":"ContainerDied","Data":"d957fbf3fc6f2cc959d504188783e5649f3067643d0ea81784e1f31f58c07b51"}
Oct 31 01:27:56.055594 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.055559    8029 scope.go:122] "RemoveContainer" containerID="0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0"
Oct 31 01:27:56.056942 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.056919    8029 generic.go:356] "Generic (PLEG): container finished" podID="0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" containerID="257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08" exitCode=0
Oct 31 01:27:56.057089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.056989    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"
Oct 31 01:27:56.057089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.056999    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs" event={"ID":"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a","Type":"ContainerDied","Data":"257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08"}
Oct 31 01:27:56.057089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.057046    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs" event={"ID":"0aba2d04-fc4a-4df9-87cb-dbf818d45c3a","Type":"ContainerDied","Data":"e0662719478f89ea703cb2d3ec23574a74a7571f45e5b409f6305b063ac0bc62"}
Oct 31 01:27:56.066822 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.066803    8029 scope.go:122] "RemoveContainer" containerID="0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0"
Oct 31 01:27:56.067191 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:27:56.067165    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0\": container with ID starting with 0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0 not found: ID does not exist" containerID="0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0"
Oct 31 01:27:56.067267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.067202    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0"} err="failed to get container status \"0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0\": rpc error: code = NotFound desc = could not find container \"0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0\": container with ID starting with 0eea5c6deb0ec71a72a9883c20681f9baf12a680a4d8b9a8580579f9b65d3ec0 not found: ID does not exist"
Oct 31 01:27:56.067267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.067220    8029 scope.go:122] "RemoveContainer" containerID="257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08"
Oct 31 01:27:56.077510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.077489    8029 scope.go:122] "RemoveContainer" containerID="257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08"
Oct 31 01:27:56.077867 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:27:56.077845    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08\": container with ID starting with 257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08 not found: ID does not exist" containerID="257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08"
Oct 31 01:27:56.077936 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.077876    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08"} err="failed to get container status \"257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08\": rpc error: code = NotFound desc = could not find container \"257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08\": container with ID starting with 257a5cf80c0da4829afc7529c760f0971ede695ff8d30f73e4cf75d347c68e08 not found: ID does not exist"
Oct 31 01:27:56.081703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.081678    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"]
Oct 31 01:27:56.092666 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.092636    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b68f43ca-599d8575ff-wt59w"]
Oct 31 01:27:56.113857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.113825    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"]
Oct 31 01:27:56.121250 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.121222    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b68f43ca-79cc47b665-g8xxs"]
Oct 31 01:27:56.664249 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.664207    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"]
Oct 31 01:27:56.664766 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:56.664552    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" containerID="cri-o://3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1" gracePeriod=30
Oct 31 01:27:57.182229 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:57.182194    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" path="/var/lib/kubelet/pods/0aba2d04-fc4a-4df9-87cb-dbf818d45c3a/volumes"
Oct 31 01:27:57.182642 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:57.182620    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f3fd78f-7888-4741-a9b2-b6d082f7fa67" path="/var/lib/kubelet/pods/1f3fd78f-7888-4741-a9b2-b6d082f7fa67/volumes"
Oct 31 01:27:58.445082 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445047    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"]
Oct 31 01:27:58.445533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445498    8029 cpu_manager.go:414] "containerMap: removing container" podUID="1f3fd78f-7888-4741-a9b2-b6d082f7fa67" containerName="payload-pre-processing"
Oct 31 01:27:58.445533 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445509    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f3fd78f-7888-4741-a9b2-b6d082f7fa67" containerName="payload-pre-processing"
Oct 31 01:27:58.445638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445540    8029 cpu_manager.go:414] "containerMap: removing container" podUID="0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" containerName="payload-processing"
Oct 31 01:27:58.445638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445545    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" containerName="payload-processing"
Oct 31 01:27:58.445638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445559    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e2b36dc5-9571-400f-8c6c-19abedb02662" containerName="revoke-keys"
Oct 31 01:27:58.445638 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445564    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b36dc5-9571-400f-8c6c-19abedb02662" containerName="revoke-keys"
Oct 31 01:27:58.445772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445649    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aba2d04-fc4a-4df9-87cb-dbf818d45c3a" containerName="payload-processing"
Oct 31 01:27:58.445772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445660    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b36dc5-9571-400f-8c6c-19abedb02662" containerName="revoke-keys"
Oct 31 01:27:58.445772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.445669    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f3fd78f-7888-4741-a9b2-b6d082f7fa67" containerName="payload-pre-processing"
Oct 31 01:27:58.450749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.450721    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.460617 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.460576    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-cec774-openshift-default-dockercfg-rbdq7\""
Oct 31 01:27:58.464906 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.464845    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"]
Oct 31 01:27:58.526696 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.526661    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.526892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.526719    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.526892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.526844    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.526892 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.526883    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvwt\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.526999 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.526948    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.527064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.527044    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.527112 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.527099    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.527145 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.527125    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.527186 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.527174    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.627897 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.627786    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.627925    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.627979    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628089 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628033    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628099    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628173    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628205    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628217 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628212    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvwt\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628419 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628260    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628419 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628356    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628488    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628495    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.628748 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628729    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.629077 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.628955    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.630832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.630810    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.631025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.631007    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.638449 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.638423    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.638693 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.638673    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvwt\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt\") pod \"e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") " pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:58.768661 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:58.768542    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:27:59.110050 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:59.110017    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"]
Oct 31 01:27:59.110359 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:27:59.110330    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9bb7b3_bd74_468b_8885_307fec47a26a.slice/crio-a98f9fd2a7dcffc8f603785b6acd41851d1cf21be6bd33f79ea915f1fade24eb WatchSource:0}: Error finding container a98f9fd2a7dcffc8f603785b6acd41851d1cf21be6bd33f79ea915f1fade24eb: Status 404 returned error can't find the container with id a98f9fd2a7dcffc8f603785b6acd41851d1cf21be6bd33f79ea915f1fade24eb
Oct 31 01:27:59.112847 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:59.112813    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:59.112976 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:59.112883    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:27:59.112976 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:27:59.112911    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:28:00.084412 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.084369    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" event={"ID":"9a9bb7b3-bd74-468b-8885-307fec47a26a","Type":"ContainerStarted","Data":"f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"}
Oct 31 01:28:00.084882 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.084455    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" event={"ID":"9a9bb7b3-bd74-468b-8885-307fec47a26a","Type":"ContainerStarted","Data":"a98f9fd2a7dcffc8f603785b6acd41851d1cf21be6bd33f79ea915f1fade24eb"}
Oct 31 01:28:00.107371 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.107314    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podStartSLOduration=2.107294717 podStartE2EDuration="2.107294717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:28:00.106862491 +0000 UTC m=+1877.093974488" watchObservedRunningTime="2026-10-31 01:28:00.107294717 +0000 UTC m=+1877.094406692"
Oct 31 01:28:00.769588 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.769549    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:00.775185 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.775154    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:00.996870 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.996823    8029 patch_prober.go:29] interesting pod/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 01:28:00.997054 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:00.996887    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 01:28:01.090012 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.089922    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:01.091113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.091091    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:01.935403 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.935328    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:28:01.964986 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.964952    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965195 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.964998    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965195 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965060    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965195 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965114    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965195 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965145    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965203    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965273    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965325    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:01.965405 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965355    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965438    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965454    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965445    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token\") pod \"dba5bc2b-8070-4321-a9ac-79b28e844103\" (UID: \"dba5bc2b-8070-4321-a9ac-79b28e844103\") "
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.965893    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.966006    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.966025    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.966039    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dba5bc2b-8070-4321-a9ac-79b28e844103-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:01.965596 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.966052    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:01.966668 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.966186    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:01.968173 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.968142    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:28:01.968374 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.968335    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "kube-api-access-7x7f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:01.968658 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.968634    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:28:01.974809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:01.974772    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token" pod "dba5bc2b-8070-4321-a9ac-79b28e844103" (UID: "dba5bc2b-8070-4321-a9ac-79b28e844103"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:28:02.067047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.067007    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:02.067047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.067040    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x7f2\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-kube-api-access-7x7f2\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:02.067047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.067051    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:02.067047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.067061    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:02.067349 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.067069    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dba5bc2b-8070-4321-a9ac-79b28e844103-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:28:02.096543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.096505    8029 generic.go:356] "Generic (PLEG): container finished" podID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerID="3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1" exitCode=0
Oct 31 01:28:02.097014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.096557    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"
Oct 31 01:28:02.097014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.096630    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" event={"ID":"dba5bc2b-8070-4321-a9ac-79b28e844103","Type":"ContainerDied","Data":"3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1"}
Oct 31 01:28:02.097014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.096691    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf" event={"ID":"dba5bc2b-8070-4321-a9ac-79b28e844103","Type":"ContainerDied","Data":"2daa9dcbc85694dcf8da4afec3aca9b5fabb31c5d1fc3001da774b3bafe8cb76"}
Oct 31 01:28:02.097014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.096719    8029 scope.go:122] "RemoveContainer" containerID="3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1"
Oct 31 01:28:02.107095 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.107071    8029 scope.go:122] "RemoveContainer" containerID="a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"
Oct 31 01:28:02.116709 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.116684    8029 scope.go:122] "RemoveContainer" containerID="3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1"
Oct 31 01:28:02.117076 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:28:02.117057    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1\": container with ID starting with 3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1 not found: ID does not exist" containerID="3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1"
Oct 31 01:28:02.117122 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.117089    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1"} err="failed to get container status \"3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1\": rpc error: code = NotFound desc = could not find container \"3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1\": container with ID starting with 3e0617a0e0a0a99c0d2a4d7fbefcc3d960ef4a44d51de7cece830d5e6580e3c1 not found: ID does not exist"
Oct 31 01:28:02.117122 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.117108    8029 scope.go:122] "RemoveContainer" containerID="a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"
Oct 31 01:28:02.117401 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:28:02.117380    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f\": container with ID starting with a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f not found: ID does not exist" containerID="a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"
Oct 31 01:28:02.117439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.117408    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f"} err="failed to get container status \"a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f\": rpc error: code = NotFound desc = could not find container \"a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f\": container with ID starting with a12666a78a64e6cfa3291ce207455fc17976e5e1640c920d6c6b6849ecd8561f not found: ID does not exist"
Oct 31 01:28:02.122562 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.122529    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"]
Oct 31 01:28:02.127032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:02.127002    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b68f43ca-openshift-default-664d8cf66c-4vmsf"]
Oct 31 01:28:03.183191 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.183156    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" path="/var/lib/kubelet/pods/dba5bc2b-8070-4321-a9ac-79b28e844103/volumes"
Oct 31 01:28:03.614282 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.614240    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"]
Oct 31 01:28:03.614818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.614798    8029 cpu_manager.go:414] "containerMap: removing container" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy"
Oct 31 01:28:03.614818 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.614819    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy"
Oct 31 01:28:03.614934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.614921    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy"
Oct 31 01:28:03.614934 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.614932    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy"
Oct 31 01:28:03.619557 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.619536    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.626423 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.626393    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-cec774-dockercfg-crbqr\""
Oct 31 01:28:03.626587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.626471    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-cec774\""
Oct 31 01:28:03.628878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.628844    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"]
Oct 31 01:28:03.652010 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.651968    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"]
Oct 31 01:28:03.652526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.652507    8029 cpu_manager.go:414] "containerMap: removing container" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy"
Oct 31 01:28:03.652526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.652525    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dba5bc2b-8070-4321-a9ac-79b28e844103" containerName="istio-proxy"
Oct 31 01:28:03.655878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.655853    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.666025 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.665990    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"]
Oct 31 01:28:03.684372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.684336    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.684615 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.684573    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wwt6\" (UniqueName: \"kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6\") pod \"payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.684708 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.684635    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.684753 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.684705    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2899\" (UniqueName: \"kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899\") pod \"payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.785969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.785921    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wwt6\" (UniqueName: \"kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6\") pod \"payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.785969 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.785977    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.786269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.786008    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2899\" (UniqueName: \"kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899\") pod \"payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.786269 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.786066    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.786819 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.786790    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.786952 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.786886    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.795153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.795124    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wwt6\" (UniqueName: \"kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6\") pod \"payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.795274 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.795251    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2899\" (UniqueName: \"kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899\") pod \"payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:03.935801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.935702    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:03.972829 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:03.972790    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:04.216933 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:04.216903    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"]
Oct 31 01:28:04.220094 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:28:04.220061    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40fbc0fd_4dc0_4898_a93e_e425c02ed0d0.slice/crio-74d4fce8dc682b84b45551e1ce80fbb11ba206ff634e61d0b22d2b40fe201f03 WatchSource:0}: Error finding container 74d4fce8dc682b84b45551e1ce80fbb11ba206ff634e61d0b22d2b40fe201f03: Status 404 returned error can't find the container with id 74d4fce8dc682b84b45551e1ce80fbb11ba206ff634e61d0b22d2b40fe201f03
Oct 31 01:28:04.282018 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:04.281657    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"]
Oct 31 01:28:04.287697 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:28:04.287664    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42552b18_ba92_4311_883a_ac055bfe5535.slice/crio-739c3bcf7d9d16caf4b71e56e72cd44cfc0b4559a490f28efa13faa2e0c0de7f WatchSource:0}: Error finding container 739c3bcf7d9d16caf4b71e56e72cd44cfc0b4559a490f28efa13faa2e0c0de7f: Status 404 returned error can't find the container with id 739c3bcf7d9d16caf4b71e56e72cd44cfc0b4559a490f28efa13faa2e0c0de7f
Oct 31 01:28:05.118520 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.118484    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d" event={"ID":"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0","Type":"ContainerStarted","Data":"0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09"}
Oct 31 01:28:05.118728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.118540    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d" event={"ID":"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0","Type":"ContainerStarted","Data":"74d4fce8dc682b84b45551e1ce80fbb11ba206ff634e61d0b22d2b40fe201f03"}
Oct 31 01:28:05.118728 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.118664    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:05.120330 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.120301    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9" event={"ID":"42552b18-ba92-4311-883a-ac055bfe5535","Type":"ContainerStarted","Data":"9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1"}
Oct 31 01:28:05.120473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.120354    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9" event={"ID":"42552b18-ba92-4311-883a-ac055bfe5535","Type":"ContainerStarted","Data":"739c3bcf7d9d16caf4b71e56e72cd44cfc0b4559a490f28efa13faa2e0c0de7f"}
Oct 31 01:28:05.120473 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.120396    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:05.136849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.136790    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d" podStartSLOduration=2.136776332 podStartE2EDuration="2.136776332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:28:05.135637417 +0000 UTC m=+1882.122749396" watchObservedRunningTime="2026-10-31 01:28:05.136776332 +0000 UTC m=+1882.123888313"
Oct 31 01:28:05.153764 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:05.153697    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9" podStartSLOduration=2.15367759 podStartE2EDuration="2.15367759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:28:05.153023781 +0000 UTC m=+1882.140135758" watchObservedRunningTime="2026-10-31 01:28:05.15367759 +0000 UTC m=+1882.140789567"
Oct 31 01:28:16.091614 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.091508    8029 patch_prober.go:29] interesting pod/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:28:16.091614 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.091579    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:28:16.129233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.129193    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:28:16.129407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.129275    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:28:16.192047 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.191949    8029 generic.go:356] "Generic (PLEG): container finished" podID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerID="f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5" exitCode=0
Oct 31 01:28:16.192210 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.192038    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" event={"ID":"9a9bb7b3-bd74-468b-8885-307fec47a26a","Type":"ContainerDied","Data":"f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"}
Oct 31 01:28:16.192584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.192566    8029 scope.go:122] "RemoveContainer" containerID="f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"
Oct 31 01:28:16.194362 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.194325    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:28:16.194439 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.194423    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:28:16.194507 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:16.194492    8029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 01:28:17.210316 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:17.210271    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" event={"ID":"9a9bb7b3-bd74-468b-8885-307fec47a26a","Type":"ContainerStarted","Data":"71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce"}
Oct 31 01:28:17.769716 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:17.769670    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:17.769963 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:17.769940    8029 patch_prober.go:29] interesting pod/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:28:17.770014 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:17.769991    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:28:18.769956 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:18.769864    8029 patch_prober.go:29] interesting pod/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:28:18.770353 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:18.769958    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:28:19.445568 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:28:19.445525    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:28:19.768995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:19.768893    8029 patch_prober.go:29] interesting pod/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:28:19.768995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:19.768945    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:28:20.769371 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:20.769327    8029 patch_prober.go:29] interesting pod/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:28:20.769849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:20.769403    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:28:21.769538 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:21.769494    8029 patch_prober.go:29] interesting pod/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 31 01:28:21.769997 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:21.769571    8029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 31 01:28:22.774132 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:22.774094    8029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:22.774694 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:22.774529    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:22.775290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:22.775273    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:28:45.973855 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:45.973819    8029 scope.go:122] "RemoveContainer" containerID="8a3e24523fba981a7c09d500cf72d878303558a6d525649b872363ea54ab3913"
Oct 31 01:28:45.983157 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:28:45.983132    8029 scope.go:122] "RemoveContainer" containerID="26de1dd3e293b8a130886ed3aa11d796e45e767e26ae42fc32bf105eb4414f12"
Oct 31 01:28:49.444442 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:28:49.444399    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:29:01.022386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:01.022342    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"]
Oct 31 01:29:01.027301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:01.027268    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-tknrv"]
Oct 31 01:29:01.182289 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:01.182252    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85768033-0b00-447c-9d8e-6779092ba3c6" path="/var/lib/kubelet/pods/85768033-0b00-447c-9d8e-6779092ba3c6/volumes"
Oct 31 01:29:19.445131 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:29:19.445088    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:29:20.023071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:20.023028    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"]
Oct 31 01:29:20.024786 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:20.024747    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-v7876"]
Oct 31 01:29:21.182290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:21.182248    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="489f90fc-ab63-4378-8e94-ca46780e7d59" path="/var/lib/kubelet/pods/489f90fc-ab63-4378-8e94-ca46780e7d59/volumes"
Oct 31 01:29:37.784291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.784244    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"]
Oct 31 01:29:37.788438 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.788408    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.794503 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.794474    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:29:37.795032 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.795004    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:29:37.795211 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.795191    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:29:37.795296 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.795252    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd9tk\""
Oct 31 01:29:37.797923 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.797897    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"]
Oct 31 01:29:37.854828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.854790    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.855017 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.854869    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm2zq\" (UniqueName: \"kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.855017 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.854923    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.955912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.955874    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm2zq\" (UniqueName: \"kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.956119 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.955942    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.956119 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.956081    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.956754 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.956727    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.958734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.958705    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:37.963849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:37.963821    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm2zq\" (UniqueName: \"kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq\") pod \"maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:38.103372 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:38.103324    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:38.384659 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:38.384624    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"]
Oct 31 01:29:38.387802 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:29:38.387770    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6f745a_64a3_4fee_a2dc_04bca7c9665b.slice/crio-09a5632a0dba58ee2a87130c5acef638a7e14e51a13780aecedfd12fae6e5831 WatchSource:0}: Error finding container 09a5632a0dba58ee2a87130c5acef638a7e14e51a13780aecedfd12fae6e5831: Status 404 returned error can't find the container with id 09a5632a0dba58ee2a87130c5acef638a7e14e51a13780aecedfd12fae6e5831
Oct 31 01:29:38.678452 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:38.678411    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb" event={"ID":"8f6f745a-64a3-4fee-a2dc-04bca7c9665b","Type":"ContainerStarted","Data":"09a5632a0dba58ee2a87130c5acef638a7e14e51a13780aecedfd12fae6e5831"}
Oct 31 01:29:39.685862 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:39.685826    8029 generic.go:356] "Generic (PLEG): container finished" podID="8f6f745a-64a3-4fee-a2dc-04bca7c9665b" containerID="29e01df7462a285b7186aa157b7cf17c0e5996bb5c18fd1099bdb725b209e3a4" exitCode=0
Oct 31 01:29:39.686278 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:39.685899    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb" event={"ID":"8f6f745a-64a3-4fee-a2dc-04bca7c9665b","Type":"ContainerDied","Data":"29e01df7462a285b7186aa157b7cf17c0e5996bb5c18fd1099bdb725b209e3a4"}
Oct 31 01:29:40.831964 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.831927    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:40.888535 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.888496    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq\" (UniqueName: \"kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq\") pod \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") "
Oct 31 01:29:40.888785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.888556    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca\") pod \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") "
Oct 31 01:29:40.888785 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.888735    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca\") pod \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\" (UID: \"8f6f745a-64a3-4fee-a2dc-04bca7c9665b\") "
Oct 31 01:29:40.889051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.889026    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca" pod "8f6f745a-64a3-4fee-a2dc-04bca7c9665b" (UID: "8f6f745a-64a3-4fee-a2dc-04bca7c9665b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:40.889171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.889154    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:40.891099 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.891075    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq" pod "8f6f745a-64a3-4fee-a2dc-04bca7c9665b" (UID: "8f6f745a-64a3-4fee-a2dc-04bca7c9665b"). InnerVolumeSpecName "kube-api-access-cm2zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:40.891099 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.891081    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca" pod "8f6f745a-64a3-4fee-a2dc-04bca7c9665b" (UID: "8f6f745a-64a3-4fee-a2dc-04bca7c9665b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:29:40.990547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.990457    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm2zq\" (UniqueName: \"kubernetes.io/projected/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-kube-api-access-cm2zq\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:40.990547 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:40.990491    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f6f745a-64a3-4fee-a2dc-04bca7c9665b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:41.698204 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:41.698170    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"
Oct 31 01:29:41.698399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:41.698199    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb" event={"ID":"8f6f745a-64a3-4fee-a2dc-04bca7c9665b","Type":"ContainerDied","Data":"09a5632a0dba58ee2a87130c5acef638a7e14e51a13780aecedfd12fae6e5831"}
Oct 31 01:29:41.698399 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:41.698264    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09a5632a0dba58ee2a87130c5acef638a7e14e51a13780aecedfd12fae6e5831"
Oct 31 01:29:47.920016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:47.919975    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"]
Oct 31 01:29:47.924235 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:47.924191    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"]
Oct 31 01:29:49.447166 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:29:49.447065    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:29:52.920828 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:52.920746    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9" podUID="42552b18-ba92-4311-883a-ac055bfe5535" containerName="payload-processing" containerID="cri-o://9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1" gracePeriod=25
Oct 31 01:29:52.925270 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:52.925199    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d" podUID="40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" containerName="payload-pre-processing" containerID="cri-o://0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09" gracePeriod=25
Oct 31 01:29:53.123480 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.123221    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"]
Oct 31 01:29:53.125558 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.125525    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-cec774-gd5mb"]
Oct 31 01:29:53.187105 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.187066    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f6f745a-64a3-4fee-a2dc-04bca7c9665b" path="/var/lib/kubelet/pods/8f6f745a-64a3-4fee-a2dc-04bca7c9665b/volumes"
Oct 31 01:29:53.199455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.199420    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:29:53.247705 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.247667    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:29:53.340199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.340156    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6\" (UniqueName: \"kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6\") pod \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") "
Oct 31 01:29:53.340373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.340209    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume\") pod \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\" (UID: \"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0\") "
Oct 31 01:29:53.340373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.340237    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume\") pod \"42552b18-ba92-4311-883a-ac055bfe5535\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") "
Oct 31 01:29:53.340373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.340346    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899\" (UniqueName: \"kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899\") pod \"42552b18-ba92-4311-883a-ac055bfe5535\" (UID: \"42552b18-ba92-4311-883a-ac055bfe5535\") "
Oct 31 01:29:53.340751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.340707    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume" pod "40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" (UID: "40fbc0fd-4dc0-4898-a93e-e425c02ed0d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:53.340751 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.340722    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume" pod "42552b18-ba92-4311-883a-ac055bfe5535" (UID: "42552b18-ba92-4311-883a-ac055bfe5535"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:29:53.342857 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.342826    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899" pod "42552b18-ba92-4311-883a-ac055bfe5535" (UID: "42552b18-ba92-4311-883a-ac055bfe5535"). InnerVolumeSpecName "kube-api-access-j2899". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:53.342949 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.342913    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6" pod "40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" (UID: "40fbc0fd-4dc0-4898-a93e-e425c02ed0d0"). InnerVolumeSpecName "kube-api-access-8wwt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:29:53.442237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.442141    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2899\" (UniqueName: \"kubernetes.io/projected/42552b18-ba92-4311-883a-ac055bfe5535-kube-api-access-j2899\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:53.442237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.442177    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wwt6\" (UniqueName: \"kubernetes.io/projected/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-kube-api-access-8wwt6\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:53.442237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.442188    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:53.442237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.442198    8029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42552b18-ba92-4311-883a-ac055bfe5535-plugins-config-volume\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:29:53.770585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.770489    8029 generic.go:356] "Generic (PLEG): container finished" podID="40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" containerID="0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09" exitCode=0
Oct 31 01:29:53.770585 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.770546    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"
Oct 31 01:29:53.770820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.770581    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d" event={"ID":"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0","Type":"ContainerDied","Data":"0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09"}
Oct 31 01:29:53.770820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.770668    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d" event={"ID":"40fbc0fd-4dc0-4898-a93e-e425c02ed0d0","Type":"ContainerDied","Data":"74d4fce8dc682b84b45551e1ce80fbb11ba206ff634e61d0b22d2b40fe201f03"}
Oct 31 01:29:53.770820 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.770701    8029 scope.go:122] "RemoveContainer" containerID="0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09"
Oct 31 01:29:53.772046 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.772019    8029 generic.go:356] "Generic (PLEG): container finished" podID="42552b18-ba92-4311-883a-ac055bfe5535" containerID="9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1" exitCode=0
Oct 31 01:29:53.772147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.772055    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9" event={"ID":"42552b18-ba92-4311-883a-ac055bfe5535","Type":"ContainerDied","Data":"9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1"}
Oct 31 01:29:53.772147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.772084    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"
Oct 31 01:29:53.772147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.772105    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9" event={"ID":"42552b18-ba92-4311-883a-ac055bfe5535","Type":"ContainerDied","Data":"739c3bcf7d9d16caf4b71e56e72cd44cfc0b4559a490f28efa13faa2e0c0de7f"}
Oct 31 01:29:53.781988 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.781968    8029 scope.go:122] "RemoveContainer" containerID="0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09"
Oct 31 01:29:53.782337 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:29:53.782311    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09\": container with ID starting with 0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09 not found: ID does not exist" containerID="0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09"
Oct 31 01:29:53.782448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.782353    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09"} err="failed to get container status \"0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09\": rpc error: code = NotFound desc = could not find container \"0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09\": container with ID starting with 0f4b3b02ea75d2f8461c8b888c59cdc9d6a559ef04a2119bd9a96abb690f2e09 not found: ID does not exist"
Oct 31 01:29:53.782448 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.782374    8029 scope.go:122] "RemoveContainer" containerID="9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1"
Oct 31 01:29:53.791675 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.791650    8029 scope.go:122] "RemoveContainer" containerID="9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1"
Oct 31 01:29:53.791971 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:29:53.791951    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1\": container with ID starting with 9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1 not found: ID does not exist" containerID="9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1"
Oct 31 01:29:53.792022 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.791981    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1"} err="failed to get container status \"9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1\": rpc error: code = NotFound desc = could not find container \"9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1\": container with ID starting with 9b6f48e5f22e407a4ec2f447a4dbca862eff57a669f81b67f76921f4ca6c09e1 not found: ID does not exist"
Oct 31 01:29:53.796576 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.796550    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"]
Oct 31 01:29:53.799450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.799427    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-cec774-544d98c547-rrc5d"]
Oct 31 01:29:53.812135 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.812111    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"]
Oct 31 01:29:53.815548 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:53.815527    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-cec774-9c7cc665c-m6br9"]
Oct 31 01:29:55.182703 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:55.182667    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" path="/var/lib/kubelet/pods/40fbc0fd-4dc0-4898-a93e-e425c02ed0d0/volumes"
Oct 31 01:29:55.183136 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:55.183073    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42552b18-ba92-4311-883a-ac055bfe5535" path="/var/lib/kubelet/pods/42552b18-ba92-4311-883a-ac055bfe5535/volumes"
Oct 31 01:29:59.805436 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:59.805392    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"]
Oct 31 01:29:59.806002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:29:59.805672    8029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy" containerID="cri-o://71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce" gracePeriod=30
Oct 31 01:30:00.144570 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.144530    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"]
Oct 31 01:30:00.145062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145043    8029 cpu_manager.go:414] "containerMap: removing container" podUID="40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" containerName="payload-pre-processing"
Oct 31 01:30:00.145062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145061    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" containerName="payload-pre-processing"
Oct 31 01:30:00.145062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145077    8029 cpu_manager.go:414] "containerMap: removing container" podUID="42552b18-ba92-4311-883a-ac055bfe5535" containerName="payload-processing"
Oct 31 01:30:00.145333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145082    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42552b18-ba92-4311-883a-ac055bfe5535" containerName="payload-processing"
Oct 31 01:30:00.145333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145095    8029 cpu_manager.go:414] "containerMap: removing container" podUID="8f6f745a-64a3-4fee-a2dc-04bca7c9665b" containerName="revoke-keys"
Oct 31 01:30:00.145333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145100    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f6f745a-64a3-4fee-a2dc-04bca7c9665b" containerName="revoke-keys"
Oct 31 01:30:00.145333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145179    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="42552b18-ba92-4311-883a-ac055bfe5535" containerName="payload-processing"
Oct 31 01:30:00.145333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145186    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f6f745a-64a3-4fee-a2dc-04bca7c9665b" containerName="revoke-keys"
Oct 31 01:30:00.145333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.145200    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="40fbc0fd-4dc0-4898-a93e-e425c02ed0d0" containerName="payload-pre-processing"
Oct 31 01:30:00.150203 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.150176    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.157836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.157805    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:30:00.158106 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.158061    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd9tk\""
Oct 31 01:30:00.158201 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.158160    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:30:00.160816 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.160476    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"]
Oct 31 01:30:00.214009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.213972    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890170-5tlcq\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.214009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.214019    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b5vl\" (UniqueName: \"kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl\") pod \"maas-api-key-cleanup-29890170-5tlcq\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.314912 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.314869    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890170-5tlcq\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.315102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.314921    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b5vl\" (UniqueName: \"kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl\") pod \"maas-api-key-cleanup-29890170-5tlcq\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.315565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.315539    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890170-5tlcq\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.328734 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.328701    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b5vl\" (UniqueName: \"kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl\") pod \"maas-api-key-cleanup-29890170-5tlcq\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.470064 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.469968    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:00.751453 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.751426    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"]
Oct 31 01:30:00.754415 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:30:00.754379    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e40545_90e6_45e2_a8a6_353a6f69e71c.slice/crio-8e3eb94208b98672bbe98954599bfacda311493e75d9079bb56caa637c9e4138 WatchSource:0}: Error finding container 8e3eb94208b98672bbe98954599bfacda311493e75d9079bb56caa637c9e4138: Status 404 returned error can't find the container with id 8e3eb94208b98672bbe98954599bfacda311493e75d9079bb56caa637c9e4138
Oct 31 01:30:00.827594 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:00.827549    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" event={"ID":"51e40545-90e6-45e2-a8a6-353a6f69e71c","Type":"ContainerStarted","Data":"8e3eb94208b98672bbe98954599bfacda311493e75d9079bb56caa637c9e4138"}
Oct 31 01:30:01.835823 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:01.835791    8029 generic.go:356] "Generic (PLEG): container finished" podID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerID="5b5ddab7cff1604d360fead26cedc56706a7c2104afec0ab0e9bc66f72f88fa5" exitCode=60
Oct 31 01:30:01.836260 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:01.835888    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" event={"ID":"51e40545-90e6-45e2-a8a6-353a6f69e71c","Type":"ContainerDied","Data":"5b5ddab7cff1604d360fead26cedc56706a7c2104afec0ab0e9bc66f72f88fa5"}
Oct 31 01:30:01.836260 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:01.836123    8029 scope.go:122] "RemoveContainer" containerID="5b5ddab7cff1604d360fead26cedc56706a7c2104afec0ab0e9bc66f72f88fa5"
Oct 31 01:30:02.847587 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:02.847488    8029 generic.go:356] "Generic (PLEG): container finished" podID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerID="ce821795c4f86070fcc4f9a4744867ffcdc303140a0c632d40bf74afbad25916" exitCode=60
Oct 31 01:30:02.848144 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:02.847592    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" event={"ID":"51e40545-90e6-45e2-a8a6-353a6f69e71c","Type":"ContainerDied","Data":"ce821795c4f86070fcc4f9a4744867ffcdc303140a0c632d40bf74afbad25916"}
Oct 31 01:30:02.848144 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:02.847689    8029 scope.go:122] "RemoveContainer" containerID="5b5ddab7cff1604d360fead26cedc56706a7c2104afec0ab0e9bc66f72f88fa5"
Oct 31 01:30:02.848261 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:02.848216    8029 scope.go:122] "RemoveContainer" containerID="ce821795c4f86070fcc4f9a4744867ffcdc303140a0c632d40bf74afbad25916"
Oct 31 01:30:02.848506 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:02.848484    8029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890170-5tlcq_odh-ai-gateway-infra(51e40545-90e6-45e2-a8a6-353a6f69e71c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c"
Oct 31 01:30:03.862232 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:03.862204    8029 scope.go:122] "RemoveContainer" containerID="ce821795c4f86070fcc4f9a4744867ffcdc303140a0c632d40bf74afbad25916"
Oct 31 01:30:03.862702 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:03.862444    8029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890170-5tlcq_odh-ai-gateway-infra(51e40545-90e6-45e2-a8a6-353a6f69e71c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c"
Oct 31 01:30:04.622758 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:04.622690    8029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-tw6hh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 01:30:04.623170 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:04.623109    8029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-tw6hh" podUID="5bc97d91-1e29-430f-b333-61b323418290" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 01:30:05.074452 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.074419    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:30:05.176290 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176244    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176318    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176350    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176407    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176444    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176510 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176476    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176512    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176554    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.176801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176682    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert\") pod \"9a9bb7b3-bd74-468b-8885-307fec47a26a\" (UID: \"9a9bb7b3-bd74-468b-8885-307fec47a26a\") "
Oct 31 01:30:05.177198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.176967    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:05.177198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.177095    8029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-credential-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.177198 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.177184    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:05.177470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.177416    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:05.177630 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.177479    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:05.177712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.177560    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:05.178957 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.178920    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 01:30:05.178957 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.178925    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 01:30:05.179551 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.179533    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:05.179674 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.179645    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt" pod "9a9bb7b3-bd74-468b-8885-307fec47a26a" (UID: "9a9bb7b3-bd74-468b-8885-307fec47a26a"). InnerVolumeSpecName "kube-api-access-5rvwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:05.278306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278262    8029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-token\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278299    8029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-certs\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278310    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rvwt\" (UniqueName: \"kubernetes.io/projected/9a9bb7b3-bd74-468b-8885-307fec47a26a-kube-api-access-5rvwt\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278306 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278319    8029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-workload-socket\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278330    8029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9bb7b3-bd74-468b-8885-307fec47a26a-istiod-ca-cert\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278339    8029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-envoy\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278348    8029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-podinfo\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.278584 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.278358    8029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9bb7b3-bd74-468b-8885-307fec47a26a-istio-data\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:05.875572 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.875533    8029 generic.go:356] "Generic (PLEG): container finished" podID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerID="71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce" exitCode=0
Oct 31 01:30:05.875809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.875624    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"
Oct 31 01:30:05.875809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.875627    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" event={"ID":"9a9bb7b3-bd74-468b-8885-307fec47a26a","Type":"ContainerDied","Data":"71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce"}
Oct 31 01:30:05.875809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.875777    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn" event={"ID":"9a9bb7b3-bd74-468b-8885-307fec47a26a","Type":"ContainerDied","Data":"a98f9fd2a7dcffc8f603785b6acd41851d1cf21be6bd33f79ea915f1fade24eb"}
Oct 31 01:30:05.875809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.875801    8029 scope.go:122] "RemoveContainer" containerID="71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce"
Oct 31 01:30:05.886097 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.886077    8029 scope.go:122] "RemoveContainer" containerID="f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"
Oct 31 01:30:05.893593 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.893558    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"]
Oct 31 01:30:05.896908 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.896871    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-cec774-openshift-default-7b968484d5-7kcdn"]
Oct 31 01:30:05.897863 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.897844    8029 scope.go:122] "RemoveContainer" containerID="71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce"
Oct 31 01:30:05.898233 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:05.898210    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce\": container with ID starting with 71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce not found: ID does not exist" containerID="71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce"
Oct 31 01:30:05.898291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.898245    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce"} err="failed to get container status \"71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce\": rpc error: code = NotFound desc = could not find container \"71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce\": container with ID starting with 71ad64ff04bf929508e7b577c3984cd2597bd5d90828062ecb8fbc6f7ccc11ce not found: ID does not exist"
Oct 31 01:30:05.898291 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.898267    8029 scope.go:122] "RemoveContainer" containerID="f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"
Oct 31 01:30:05.898564 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:05.898543    8029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5\": container with ID starting with f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5 not found: ID does not exist" containerID="f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"
Oct 31 01:30:05.898673 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:05.898574    8029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5"} err="failed to get container status \"f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5\": rpc error: code = NotFound desc = could not find container \"f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5\": container with ID starting with f3c1fa1afed9b909c9fc14db297b1e03babfa518b5beac9a5b370976d165bea5 not found: ID does not exist"
Oct 31 01:30:07.182553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:07.182507    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" path="/var/lib/kubelet/pods/9a9bb7b3-bd74-468b-8885-307fec47a26a/volumes"
Oct 31 01:30:11.553443 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.553403    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"]
Oct 31 01:30:11.554016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.553926    8029 cpu_manager.go:414] "containerMap: removing container" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy"
Oct 31 01:30:11.554016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.553941    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy"
Oct 31 01:30:11.554016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.553968    8029 cpu_manager.go:414] "containerMap: removing container" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy"
Oct 31 01:30:11.554016 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.553974    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy"
Oct 31 01:30:11.554155 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.554058    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy"
Oct 31 01:30:11.558773 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.558748    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.562565 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.562536    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:30:11.564461 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.564429    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"]
Oct 31 01:30:11.572738 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.572686    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.573031 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.572988    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.573158 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.573089    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dfqb\" (UniqueName: \"kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.674553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.674507    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.674553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.674553    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.674884 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.674617    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dfqb\" (UniqueName: \"kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.675334 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.675301    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.677391 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.677362    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.682878 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.682845    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dfqb\" (UniqueName: \"kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb\") pod \"maas-api-revoke-sub-f03629db-qghm6\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:11.877707 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:11.877663    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:12.156123 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:12.156091    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"]
Oct 31 01:30:12.159108 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:30:12.159076    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b6b708f_a4f9_4a01_8327_eda300f38d1b.slice/crio-d8b68c6ef9dd099c19e41a6e92178b572a50c1c530e1fff0165a30dc75bd66bd WatchSource:0}: Error finding container d8b68c6ef9dd099c19e41a6e92178b572a50c1c530e1fff0165a30dc75bd66bd: Status 404 returned error can't find the container with id d8b68c6ef9dd099c19e41a6e92178b572a50c1c530e1fff0165a30dc75bd66bd
Oct 31 01:30:12.965137 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:12.965097    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6" event={"ID":"6b6b708f-a4f9-4a01-8327-eda300f38d1b","Type":"ContainerStarted","Data":"fac9934d7c3e2cf25979754ae25e20341080180cd534db2f795e669544a5a219"}
Oct 31 01:30:12.965137 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:12.965152    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6" event={"ID":"6b6b708f-a4f9-4a01-8327-eda300f38d1b","Type":"ContainerStarted","Data":"d8b68c6ef9dd099c19e41a6e92178b572a50c1c530e1fff0165a30dc75bd66bd"}
Oct 31 01:30:13.972665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:13.972622    8029 generic.go:356] "Generic (PLEG): container finished" podID="6b6b708f-a4f9-4a01-8327-eda300f38d1b" containerID="fac9934d7c3e2cf25979754ae25e20341080180cd534db2f795e669544a5a219" exitCode=0
Oct 31 01:30:13.973184 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:13.972700    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6" event={"ID":"6b6b708f-a4f9-4a01-8327-eda300f38d1b","Type":"ContainerDied","Data":"fac9934d7c3e2cf25979754ae25e20341080180cd534db2f795e669544a5a219"}
Oct 31 01:30:15.134881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.134846    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:15.211508 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.211454    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca\") pod \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") "
Oct 31 01:30:15.211508 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.211506    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca\") pod \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") "
Oct 31 01:30:15.211826 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.211675    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb\" (UniqueName: \"kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb\") pod \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\" (UID: \"6b6b708f-a4f9-4a01-8327-eda300f38d1b\") "
Oct 31 01:30:15.211944 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.211919    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca" pod "6b6b708f-a4f9-4a01-8327-eda300f38d1b" (UID: "6b6b708f-a4f9-4a01-8327-eda300f38d1b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:15.212085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.212069    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:15.214051 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.214016    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca" pod "6b6b708f-a4f9-4a01-8327-eda300f38d1b" (UID: "6b6b708f-a4f9-4a01-8327-eda300f38d1b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:15.214206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.214078    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb" pod "6b6b708f-a4f9-4a01-8327-eda300f38d1b" (UID: "6b6b708f-a4f9-4a01-8327-eda300f38d1b"). InnerVolumeSpecName "kube-api-access-5dfqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:15.313801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.313695    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dfqb\" (UniqueName: \"kubernetes.io/projected/6b6b708f-a4f9-4a01-8327-eda300f38d1b-kube-api-access-5dfqb\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:15.313801 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.313745    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6b6b708f-a4f9-4a01-8327-eda300f38d1b-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:15.989193 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.989158    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6"
Oct 31 01:30:15.989400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.989194    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-qghm6" event={"ID":"6b6b708f-a4f9-4a01-8327-eda300f38d1b","Type":"ContainerDied","Data":"d8b68c6ef9dd099c19e41a6e92178b572a50c1c530e1fff0165a30dc75bd66bd"}
Oct 31 01:30:15.989400 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:15.989272    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8b68c6ef9dd099c19e41a6e92178b572a50c1c530e1fff0165a30dc75bd66bd"
Oct 31 01:30:16.176407 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:16.176375    8029 scope.go:122] "RemoveContainer" containerID="ce821795c4f86070fcc4f9a4744867ffcdc303140a0c632d40bf74afbad25916"
Oct 31 01:30:16.998664 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:16.998631    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" event={"ID":"51e40545-90e6-45e2-a8a6-353a6f69e71c","Type":"ContainerStarted","Data":"6cf2c3731c71a1352b50a7bef383648f6c00b0b0bc56c2785a096ec82d16817d"}
Oct 31 01:30:16.999026 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:16.999004    8029 scope.go:122] "RemoveContainer" containerID="6cf2c3731c71a1352b50a7bef383648f6c00b0b0bc56c2785a096ec82d16817d"
Oct 31 01:30:16.999304 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:16.999283    8029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890170-5tlcq_odh-ai-gateway-infra(51e40545-90e6-45e2-a8a6-353a6f69e71c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c"
Oct 31 01:30:18.005916 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:18.005881    8029 generic.go:356] "Generic (PLEG): container finished" podID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerID="6cf2c3731c71a1352b50a7bef383648f6c00b0b0bc56c2785a096ec82d16817d" exitCode=60
Oct 31 01:30:18.006398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:18.005954    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" event={"ID":"51e40545-90e6-45e2-a8a6-353a6f69e71c","Type":"ContainerDied","Data":"6cf2c3731c71a1352b50a7bef383648f6c00b0b0bc56c2785a096ec82d16817d"}
Oct 31 01:30:18.006398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:18.006018    8029 scope.go:122] "RemoveContainer" containerID="ce821795c4f86070fcc4f9a4744867ffcdc303140a0c632d40bf74afbad25916"
Oct 31 01:30:18.006398 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:18.006375    8029 scope.go:122] "RemoveContainer" containerID="6cf2c3731c71a1352b50a7bef383648f6c00b0b0bc56c2785a096ec82d16817d"
Oct 31 01:30:18.006706 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:18.006674    8029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890170-5tlcq_odh-ai-gateway-infra(51e40545-90e6-45e2-a8a6-353a6f69e71c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c"
Oct 31 01:30:18.029720 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:18.029681    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"]
Oct 31 01:30:19.169623 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.169579    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:19.256836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.256779    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca\") pod \"51e40545-90e6-45e2-a8a6-353a6f69e71c\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") "
Oct 31 01:30:19.257056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.256907    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl\" (UniqueName: \"kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl\") pod \"51e40545-90e6-45e2-a8a6-353a6f69e71c\" (UID: \"51e40545-90e6-45e2-a8a6-353a6f69e71c\") "
Oct 31 01:30:19.257243 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.257213    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca" pod "51e40545-90e6-45e2-a8a6-353a6f69e71c" (UID: "51e40545-90e6-45e2-a8a6-353a6f69e71c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:19.257853 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.257748    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e40545-90e6-45e2-a8a6-353a6f69e71c-maas-api-service-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:19.259314 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.259288    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl" pod "51e40545-90e6-45e2-a8a6-353a6f69e71c" (UID: "51e40545-90e6-45e2-a8a6-353a6f69e71c"). InnerVolumeSpecName "kube-api-access-5b5vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:19.359308 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:19.359259    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b5vl\" (UniqueName: \"kubernetes.io/projected/51e40545-90e6-45e2-a8a6-353a6f69e71c-kube-api-access-5b5vl\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:19.444256 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:19.444208    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:30:20.021045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:20.021010    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"
Oct 31 01:30:20.021264 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:20.021006    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq" event={"ID":"51e40545-90e6-45e2-a8a6-353a6f69e71c","Type":"ContainerDied","Data":"8e3eb94208b98672bbe98954599bfacda311493e75d9079bb56caa637c9e4138"}
Oct 31 01:30:20.021264 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:20.021161    8029 scope.go:122] "RemoveContainer" containerID="6cf2c3731c71a1352b50a7bef383648f6c00b0b0bc56c2785a096ec82d16817d"
Oct 31 01:30:20.044133 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:20.044092    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"]
Oct 31 01:30:20.049754 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:20.049714    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890170-5tlcq"]
Oct 31 01:30:21.182341 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:21.182301    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" path="/var/lib/kubelet/pods/51e40545-90e6-45e2-a8a6-353a6f69e71c/volumes"
Oct 31 01:30:24.043263 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:24.043228    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"]
Oct 31 01:30:24.053746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:24.053712    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7648cda3-l4f6c"]
Oct 31 01:30:25.212134 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:25.212074    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="018b278a-01ce-4bbd-90ca-ee94799712fb" path="/var/lib/kubelet/pods/018b278a-01ce-4bbd-90ca-ee94799712fb/volumes"
Oct 31 01:30:27.383223 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383173    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"]
Oct 31 01:30:27.383680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383655    8029 cpu_manager.go:414] "containerMap: removing container" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:27.383680 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383669    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:27.383763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383700    8029 cpu_manager.go:414] "containerMap: removing container" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:27.383763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383705    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:27.383763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383713    8029 cpu_manager.go:414] "containerMap: removing container" podUID="6b6b708f-a4f9-4a01-8327-eda300f38d1b" containerName="revoke-keys"
Oct 31 01:30:27.383763 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383720    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b6b708f-a4f9-4a01-8327-eda300f38d1b" containerName="revoke-keys"
Oct 31 01:30:27.383890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383786    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:27.383890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383794    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b6b708f-a4f9-4a01-8327-eda300f38d1b" containerName="revoke-keys"
Oct 31 01:30:27.383890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383803    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a9bb7b3-bd74-468b-8885-307fec47a26a" containerName="istio-proxy"
Oct 31 01:30:27.383890 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.383811    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:27.388365 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.388342    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.392926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.392868    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:30:27.392926 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.392888    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:30:27.393189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.392933    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd9tk\""
Oct 31 01:30:27.393417 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.393387    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:30:27.395308 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.395280    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"]
Oct 31 01:30:27.475151 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.475113    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vd24\" (UniqueName: \"kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.475386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.475241    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.475386 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.475286    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.576614 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.576566    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vd24\" (UniqueName: \"kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.576837 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.576735    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.576837 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.576763    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.577447 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.577422    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.579492 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.579465    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.585304 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.585272    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vd24\" (UniqueName: \"kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24\") pod \"maas-api-revoke-sub-0165d7d2-tp24f\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.706835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.706739    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:27.984719 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:27.984593    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"]
Oct 31 01:30:27.987729 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:30:27.987694    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541a17b3_7ccc_49b9_b387_179a9659a900.slice/crio-1fc9dd575f46d8df6a39792b902a7f311c0cffd39f78984d35b5f2bf8efb5a54 WatchSource:0}: Error finding container 1fc9dd575f46d8df6a39792b902a7f311c0cffd39f78984d35b5f2bf8efb5a54: Status 404 returned error can't find the container with id 1fc9dd575f46d8df6a39792b902a7f311c0cffd39f78984d35b5f2bf8efb5a54
Oct 31 01:30:28.072976 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:28.072931    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f" event={"ID":"541a17b3-7ccc-49b9-b387-179a9659a900","Type":"ContainerStarted","Data":"1fc9dd575f46d8df6a39792b902a7f311c0cffd39f78984d35b5f2bf8efb5a54"}
Oct 31 01:30:29.080216 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:29.080116    8029 generic.go:356] "Generic (PLEG): container finished" podID="541a17b3-7ccc-49b9-b387-179a9659a900" containerID="5c3ef097f6676ba85965a9204d411b0890effbdbcc55db921a5f4db0dc97c6e5" exitCode=0
Oct 31 01:30:29.080216 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:29.080197    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f" event={"ID":"541a17b3-7ccc-49b9-b387-179a9659a900","Type":"ContainerDied","Data":"5c3ef097f6676ba85965a9204d411b0890effbdbcc55db921a5f4db0dc97c6e5"}
Oct 31 01:30:30.229481 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.229441    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:30.307616 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.307562    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca\") pod \"541a17b3-7ccc-49b9-b387-179a9659a900\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") "
Oct 31 01:30:30.307835 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.307720    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca\") pod \"541a17b3-7ccc-49b9-b387-179a9659a900\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") "
Oct 31 01:30:30.307922 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.307835    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24\" (UniqueName: \"kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24\") pod \"541a17b3-7ccc-49b9-b387-179a9659a900\" (UID: \"541a17b3-7ccc-49b9-b387-179a9659a900\") "
Oct 31 01:30:30.308113 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.308087    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca" pod "541a17b3-7ccc-49b9-b387-179a9659a900" (UID: "541a17b3-7ccc-49b9-b387-179a9659a900"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:30.308237 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.308220    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:30.309991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.309947    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca" pod "541a17b3-7ccc-49b9-b387-179a9659a900" (UID: "541a17b3-7ccc-49b9-b387-179a9659a900"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:30.310159 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.310133    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24" pod "541a17b3-7ccc-49b9-b387-179a9659a900" (UID: "541a17b3-7ccc-49b9-b387-179a9659a900"). InnerVolumeSpecName "kube-api-access-7vd24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:30.409006 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.408963    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vd24\" (UniqueName: \"kubernetes.io/projected/541a17b3-7ccc-49b9-b387-179a9659a900-kube-api-access-7vd24\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:30.409006 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:30.408998    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/541a17b3-7ccc-49b9-b387-179a9659a900-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:31.100072 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:31.100026    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f"
Oct 31 01:30:31.100271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:31.100068    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-tp24f" event={"ID":"541a17b3-7ccc-49b9-b387-179a9659a900","Type":"ContainerDied","Data":"1fc9dd575f46d8df6a39792b902a7f311c0cffd39f78984d35b5f2bf8efb5a54"}
Oct 31 01:30:31.100271 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:31.100149    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fc9dd575f46d8df6a39792b902a7f311c0cffd39f78984d35b5f2bf8efb5a54"
Oct 31 01:30:38.099722 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.099682    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"]
Oct 31 01:30:38.100293 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.100275    8029 cpu_manager.go:414] "containerMap: removing container" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:38.100343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.100296    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:38.100343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.100313    8029 cpu_manager.go:414] "containerMap: removing container" podUID="541a17b3-7ccc-49b9-b387-179a9659a900" containerName="revoke-keys"
Oct 31 01:30:38.100343 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.100320    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="541a17b3-7ccc-49b9-b387-179a9659a900" containerName="revoke-keys"
Oct 31 01:30:38.100437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.100401    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="541a17b3-7ccc-49b9-b387-179a9659a900" containerName="revoke-keys"
Oct 31 01:30:38.100437 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.100414    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="51e40545-90e6-45e2-a8a6-353a6f69e71c" containerName="cleanup"
Oct 31 01:30:38.104092 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.104065    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.109928 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.109393    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 01:30:38.110136 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.110111    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 01:30:38.110224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.110179    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 01:30:38.110224 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.110209    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nd9tk\""
Oct 31 01:30:38.112852 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.112816    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"]
Oct 31 01:30:38.292534 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.292488    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.292534 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.292534    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7zgj\" (UniqueName: \"kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.292866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.292838    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.394059 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.394013    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.394288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.394081    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.394288 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.394100    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7zgj\" (UniqueName: \"kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.394896 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.394874    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.396995 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.396972    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.402033 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.402000    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7zgj\" (UniqueName: \"kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj\") pod \"maas-api-revoke-sub-6b19338f-2bwjj\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.424532 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.424476    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:38.712983 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:38.712879    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"]
Oct 31 01:30:38.718383 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:30:38.718341    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a49a3d_26ee_45dc_bb97_a57197b832c2.slice/crio-9f51c4389cf3d3bfea41f7106b3869ee1edb1456de2cf6b730c29d71eae95eaf WatchSource:0}: Error finding container 9f51c4389cf3d3bfea41f7106b3869ee1edb1456de2cf6b730c29d71eae95eaf: Status 404 returned error can't find the container with id 9f51c4389cf3d3bfea41f7106b3869ee1edb1456de2cf6b730c29d71eae95eaf
Oct 31 01:30:39.149153 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:39.149104    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj" event={"ID":"e7a49a3d-26ee-45dc-bb97-a57197b832c2","Type":"ContainerStarted","Data":"e2dcf7bbcb94691c94aa3147648d2554168663b0aa9470e8ffb00a4531b61090"}
Oct 31 01:30:39.149665 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:39.149169    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj" event={"ID":"e7a49a3d-26ee-45dc-bb97-a57197b832c2","Type":"ContainerStarted","Data":"9f51c4389cf3d3bfea41f7106b3869ee1edb1456de2cf6b730c29d71eae95eaf"}
Oct 31 01:30:39.165941 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:39.165877    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj" podStartSLOduration=1.165848327 podStartE2EDuration="1.165848327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:30:39.165830953 +0000 UTC m=+2036.152942928" watchObservedRunningTime="2026-10-31 01:30:39.165848327 +0000 UTC m=+2036.152960302"
Oct 31 01:30:40.156430 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:40.156393    8029 generic.go:356] "Generic (PLEG): container finished" podID="e7a49a3d-26ee-45dc-bb97-a57197b832c2" containerID="e2dcf7bbcb94691c94aa3147648d2554168663b0aa9470e8ffb00a4531b61090" exitCode=0
Oct 31 01:30:40.156904 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:40.156448    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj" event={"ID":"e7a49a3d-26ee-45dc-bb97-a57197b832c2","Type":"ContainerDied","Data":"e2dcf7bbcb94691c94aa3147648d2554168663b0aa9470e8ffb00a4531b61090"}
Oct 31 01:30:41.306445 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.306410    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:41.323189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.323150    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj\" (UniqueName: \"kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj\") pod \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") "
Oct 31 01:30:41.323404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.323216    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca\") pod \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") "
Oct 31 01:30:41.323404 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.323246    8029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca\") pod \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\" (UID: \"e7a49a3d-26ee-45dc-bb97-a57197b832c2\") "
Oct 31 01:30:41.324273 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.324242    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca" pod "e7a49a3d-26ee-45dc-bb97-a57197b832c2" (UID: "e7a49a3d-26ee-45dc-bb97-a57197b832c2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 01:30:41.325991 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.325960    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj" pod "e7a49a3d-26ee-45dc-bb97-a57197b832c2" (UID: "e7a49a3d-26ee-45dc-bb97-a57197b832c2"). InnerVolumeSpecName "kube-api-access-q7zgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 01:30:41.326228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.326203    8029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca" pod "e7a49a3d-26ee-45dc-bb97-a57197b832c2" (UID: "e7a49a3d-26ee-45dc-bb97-a57197b832c2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 01:30:41.424971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.424860    8029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7zgj\" (UniqueName: \"kubernetes.io/projected/e7a49a3d-26ee-45dc-bb97-a57197b832c2-kube-api-access-q7zgj\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:41.424971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.424905    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-xks-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:41.424971 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:41.424918    8029 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e7a49a3d-26ee-45dc-bb97-a57197b832c2-maas-api-openshift-ca\") on node \"ip-10-0-140-244.us-east-2.compute.internal\" DevicePath \"\""
Oct 31 01:30:42.169303 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:42.169270    8029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj"
Oct 31 01:30:42.169532 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:42.169296    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2bwjj" event={"ID":"e7a49a3d-26ee-45dc-bb97-a57197b832c2","Type":"ContainerDied","Data":"9f51c4389cf3d3bfea41f7106b3869ee1edb1456de2cf6b730c29d71eae95eaf"}
Oct 31 01:30:42.169532 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:42.169373    8029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f51c4389cf3d3bfea41f7106b3869ee1edb1456de2cf6b730c29d71eae95eaf"
Oct 31 01:30:46.031840 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:46.031803    8029 scope.go:122] "RemoveContainer" containerID="cc022dfdbf76ac19bb4c63fb7027dec078df30b71306bd6ff304a88503480e89"
Oct 31 01:30:46.042409 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:46.042378    8029 scope.go:122] "RemoveContainer" containerID="8bd86fa8052669e8f55be26caa0ae95e205bb49a30c465479cdc6fe307ebd64f"
Oct 31 01:30:46.051851 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:46.051824    8029 scope.go:122] "RemoveContainer" containerID="68f34c1c85e028e73ae00b33c533a8ba6581302d0b5e6ec05f14d03e372ecdfe"
Oct 31 01:30:49.385500 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:30:49.385455    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-669748c564-fn8th_83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b/authorino/0.log"
Oct 31 01:30:49.446672 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:30:49.446621    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:31:01.242974 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:01.242942    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmb5t_65e086a0-1eb5-46cf-895e-8a9cd892ce6b/server/0.log"
Oct 31 01:31:01.442327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:01.442291    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-nmdfz_3c50d69b-8557-482a-8329-976d4556bf95/manager/0.log"
Oct 31 01:31:02.793325 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:02.793282    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-tp24f_541a17b3-7ccc-49b9-b387-179a9659a900/revoke-keys/0.log"
Oct 31 01:31:02.982695 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:02.982667    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-05301312-z5tcs_889079e9-4c68-466e-af06-e9f7073e9dea/revoke-keys/0.log"
Oct 31 01:31:03.167381 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:03.167349    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-4ndrk_a9774d05-55fd-4a32-8509-eb125fc6f8ec/revoke-keys/0.log"
Oct 31 01:31:03.343472 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:03.343442    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-hfc6h_7b35cc54-c378-4f12-a45c-5beaf665c60d/revoke-keys/0.log"
Oct 31 01:31:03.525307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:03.525229    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-2bwjj_e7a49a3d-26ee-45dc-bb97-a57197b832c2/revoke-keys/0.log"
Oct 31 01:31:03.718009 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:03.717981    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-qghm6_6b6b708f-a4f9-4a01-8327-eda300f38d1b/revoke-keys/0.log"
Oct 31 01:31:05.507102 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:05.507070    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-669748c564-fn8th_83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b/authorino/0.log"
Oct 31 01:31:07.808500 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:07.808468    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-858778b8-88xjg_1e413599-cc6e-4805-85f4-c27406052466/kube-auth-proxy/0.log"
Oct 31 01:31:09.122754 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:09.122723    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-qlmqk_d6fd01e9-d8ad-4360-8648-e90ff85643b7/istio-proxy/0.log"
Oct 31 01:31:09.311351 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:09.311320    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-sds66_38590073-ac40-4f8e-8295-5bc08ebc313b/istio-proxy/0.log"
Oct 31 01:31:09.492002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:09.491926    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-tw6hh_5bc97d91-1e29-430f-b333-61b323418290/istio-proxy/0.log"
Oct 31 01:31:11.666990 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:11.666953    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz_f13faa59-6cd2-4a45-818a-bba8f0daaaad/main/0.log"
Oct 31 01:31:12.212333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:12.212301    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-bqpc7_78f6922d-f4aa-453c-a58a-7bb0f6cc496c/main/0.log"
Oct 31 01:31:12.388806 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:12.388770    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq_4ad09d17-5b0f-4b20-9e81-7c0faf104aae/main/0.log"
Oct 31 01:31:12.719123 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:12.719091    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g_0208e85e-36ac-4a08-be16-15738c866d3e/main/0.log"
Oct 31 01:31:13.778556 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:13.778522    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-669748c564-fn8th_83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b/authorino/0.log"
Oct 31 01:31:19.443993 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:31:19.443893    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:31:23.888543 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:23.888509    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmb5t_65e086a0-1eb5-46cf-895e-8a9cd892ce6b/server/0.log"
Oct 31 01:31:24.075750 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:24.075708    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-nmdfz_3c50d69b-8557-482a-8329-976d4556bf95/manager/0.log"
Oct 31 01:31:25.216463 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:25.216427    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-tp24f_541a17b3-7ccc-49b9-b387-179a9659a900/revoke-keys/0.log"
Oct 31 01:31:25.407896 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:25.407849    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-05301312-z5tcs_889079e9-4c68-466e-af06-e9f7073e9dea/revoke-keys/0.log"
Oct 31 01:31:25.593792 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:25.593695    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-4ndrk_a9774d05-55fd-4a32-8509-eb125fc6f8ec/revoke-keys/0.log"
Oct 31 01:31:25.778147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:25.778103    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-hfc6h_7b35cc54-c378-4f12-a45c-5beaf665c60d/revoke-keys/0.log"
Oct 31 01:31:25.966328 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:25.966298    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-2bwjj_e7a49a3d-26ee-45dc-bb97-a57197b832c2/revoke-keys/0.log"
Oct 31 01:31:26.150002 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:26.149961    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-qghm6_6b6b708f-a4f9-4a01-8327-eda300f38d1b/revoke-keys/0.log"
Oct 31 01:31:27.499762 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:27.499711    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-669748c564-fn8th_83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b/authorino/0.log"
Oct 31 01:31:29.624832 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:29.624799    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-858778b8-88xjg_1e413599-cc6e-4805-85f4-c27406052466/kube-auth-proxy/0.log"
Oct 31 01:31:30.023715 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:30.023568    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"]
Oct 31 01:31:30.026810 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:30.026766    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-hfc6h"]
Oct 31 01:31:30.896228 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:30.896194    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-qlmqk_d6fd01e9-d8ad-4360-8648-e90ff85643b7/istio-proxy/0.log"
Oct 31 01:31:31.071749 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:31.071714    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-sds66_38590073-ac40-4f8e-8295-5bc08ebc313b/istio-proxy/0.log"
Oct 31 01:31:31.183165 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:31.183082    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b35cc54-c378-4f12-a45c-5beaf665c60d" path="/var/lib/kubelet/pods/7b35cc54-c378-4f12-a45c-5beaf665c60d/volumes"
Oct 31 01:31:31.253836 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:31.253805    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-tw6hh_5bc97d91-1e29-430f-b333-61b323418290/istio-proxy/0.log"
Oct 31 01:31:32.881156 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:32.881119    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-ms9dz_f13faa59-6cd2-4a45-818a-bba8f0daaaad/main/0.log"
Oct 31 01:31:33.411746 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:33.411714    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-bqpc7_78f6922d-f4aa-453c-a58a-7bb0f6cc496c/main/0.log"
Oct 31 01:31:33.593788 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:33.593746    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kqdpq_4ad09d17-5b0f-4b20-9e81-7c0faf104aae/main/0.log"
Oct 31 01:31:33.956736 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:33.956705    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-tx89g_0208e85e-36ac-4a08-be16-15738c866d3e/main/0.log"
Oct 31 01:31:43.534062 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:43.534032    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:31:43.539743 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:43.539713    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2z92c_90525854-b0fe-4589-8f36-df21ad285d0b/ovn-acl-logging/0.log"
Oct 31 01:31:45.031233 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:45.031188    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"]
Oct 31 01:31:45.031866 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:45.031844    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-4ndrk"]
Oct 31 01:31:45.182361 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:45.182323    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9774d05-55fd-4a32-8509-eb125fc6f8ec" path="/var/lib/kubelet/pods/a9774d05-55fd-4a32-8509-eb125fc6f8ec/volumes"
Oct 31 01:31:46.105554 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:46.105520    8029 scope.go:122] "RemoveContainer" containerID="3593da94beaac24407e533bee917df597eb8f966015518c968ab6f935b8fa5a1"
Oct 31 01:31:46.115147 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:31:46.115120    8029 scope.go:122] "RemoveContainer" containerID="94c2ed3afcb1a027e020f662dfe366765a59b5dcd007977ec09dc6c77883bc62"
Oct 31 01:31:49.445105 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:31:49.445068    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:32:05.019071 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:32:05.019030    8029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"]
Oct 31 01:32:05.020624 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:32:05.020586    8029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05301312-z5tcs"]
Oct 31 01:32:05.181972 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:32:05.181934    8029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="889079e9-4c68-466e-af06-e9f7073e9dea" path="/var/lib/kubelet/pods/889079e9-4c68-466e-af06-e9f7073e9dea/volumes"
Oct 31 01:32:19.444871 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:32:19.444833    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:32:46.143843 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:32:46.143807    8029 scope.go:122] "RemoveContainer" containerID="ad37cde945cf651847a7669ed3ed8a05594baa064ca477080a93028911c09fe6"
Oct 31 01:32:46.155685 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:32:46.155638    8029 scope.go:122] "RemoveContainer" containerID="fd4aaf8d68804b878831415d98c859c72f13419ed65f71aa29fb4d357cb19359"
Oct 31 01:32:49.445648 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:32:49.445519    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:33:01.715815 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.715773    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fsd8j/must-gather-tkk8s"]
Oct 31 01:33:01.716287 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.716270    8029 cpu_manager.go:414] "containerMap: removing container" podUID="e7a49a3d-26ee-45dc-bb97-a57197b832c2" containerName="revoke-keys"
Oct 31 01:33:01.716333 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.716291    8029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7a49a3d-26ee-45dc-bb97-a57197b832c2" containerName="revoke-keys"
Oct 31 01:33:01.716422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.716415    8029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7a49a3d-26ee-45dc-bb97-a57197b832c2" containerName="revoke-keys"
Oct 31 01:33:01.719753 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.719730    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:01.725729 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.725695    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fsd8j\"/\"kube-root-ca.crt\""
Oct 31 01:33:01.725895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.725740    8029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fsd8j\"/\"default-dockercfg-mphnn\""
Oct 31 01:33:01.725895 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.725800    8029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fsd8j\"/\"openshift-service-ca.crt\""
Oct 31 01:33:01.730373 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.730348    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hsl4\" (UniqueName: \"kubernetes.io/projected/187eac1a-2f63-493a-9800-497bfaf3da45-kube-api-access-6hsl4\") pod \"must-gather-tkk8s\" (UID: \"187eac1a-2f63-493a-9800-497bfaf3da45\") " pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:01.730545 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.730381    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/187eac1a-2f63-493a-9800-497bfaf3da45-must-gather-output\") pod \"must-gather-tkk8s\" (UID: \"187eac1a-2f63-493a-9800-497bfaf3da45\") " pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:01.737150 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.737117    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fsd8j/must-gather-tkk8s"]
Oct 31 01:33:01.831352 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.831301    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hsl4\" (UniqueName: \"kubernetes.io/projected/187eac1a-2f63-493a-9800-497bfaf3da45-kube-api-access-6hsl4\") pod \"must-gather-tkk8s\" (UID: \"187eac1a-2f63-493a-9800-497bfaf3da45\") " pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:01.831561 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.831360    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/187eac1a-2f63-493a-9800-497bfaf3da45-must-gather-output\") pod \"must-gather-tkk8s\" (UID: \"187eac1a-2f63-493a-9800-497bfaf3da45\") " pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:01.831837 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.831813    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/187eac1a-2f63-493a-9800-497bfaf3da45-must-gather-output\") pod \"must-gather-tkk8s\" (UID: \"187eac1a-2f63-493a-9800-497bfaf3da45\") " pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:01.840663 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:01.840590    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hsl4\" (UniqueName: \"kubernetes.io/projected/187eac1a-2f63-493a-9800-497bfaf3da45-kube-api-access-6hsl4\") pod \"must-gather-tkk8s\" (UID: \"187eac1a-2f63-493a-9800-497bfaf3da45\") " pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:02.034527 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:02.034437    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fsd8j/must-gather-tkk8s"
Oct 31 01:33:02.314712 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:02.314658    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fsd8j/must-gather-tkk8s"]
Oct 31 01:33:02.318024 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:33:02.317985    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod187eac1a_2f63_493a_9800_497bfaf3da45.slice/crio-b6604d304afdb028144fd6220e8af072ec08a1f05e5e5c3a95abe27163cab10c WatchSource:0}: Error finding container b6604d304afdb028144fd6220e8af072ec08a1f05e5e5c3a95abe27163cab10c: Status 404 returned error can't find the container with id b6604d304afdb028144fd6220e8af072ec08a1f05e5e5c3a95abe27163cab10c
Oct 31 01:33:02.320104 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:02.320082    8029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 01:33:02.951045 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:02.951004    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fsd8j/must-gather-tkk8s" event={"ID":"187eac1a-2f63-493a-9800-497bfaf3da45","Type":"ContainerStarted","Data":"b6604d304afdb028144fd6220e8af072ec08a1f05e5e5c3a95abe27163cab10c"}
Oct 31 01:33:03.969789 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:03.969744    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fsd8j/must-gather-tkk8s" event={"ID":"187eac1a-2f63-493a-9800-497bfaf3da45","Type":"ContainerStarted","Data":"66c78695722dbf3df520ebf0c59c8c75ceab33b43ba5a270ec507c2cf1dd1596"}
Oct 31 01:33:03.970345 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:03.969804    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fsd8j/must-gather-tkk8s" event={"ID":"187eac1a-2f63-493a-9800-497bfaf3da45","Type":"ContainerStarted","Data":"32dddf0ac0b713e1bb9c33e788ed3810ca843708c6e534d0b23667ec27ceac39"}
Oct 31 01:33:03.986336 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:03.986261    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-fsd8j/must-gather-tkk8s" podStartSLOduration=2.135240968 podStartE2EDuration="2.986239628s" totalImagesPullingTime="850.99866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 01:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:33:03.985920043 +0000 UTC m=+2180.973032020" watchObservedRunningTime="2026-10-31 01:33:03.986239628 +0000 UTC m=+2180.973351605"
Oct 31 01:33:05.205996 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:05.205946    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6tsqv_633346a3-a449-448c-8580-b87b22a05979/global-pull-secret-syncer/0.log"
Oct 31 01:33:05.519105 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:05.519018    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m9m29_852b2fe1-0192-404e-9c39-e16b622a3b9f/konnectivity-agent/0.log"
Oct 31 01:33:05.600470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:05.600430    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-244.us-east-2.compute.internal_751e5bfb25820222e24aac2c7122de2e/haproxy/0.log"
Oct 31 01:33:12.909498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:12.909460    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-669748c564-fn8th_83fe9f2a-bb1d-4f8b-9e1a-dce0ff96204b/authorino/0.log"
Oct 31 01:33:14.834833 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:14.834796    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-79z4l_eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/kube-state-metrics/0.log"
Oct 31 01:33:14.857475 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:14.857444    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-79z4l_eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/kube-rbac-proxy-main/0.log"
Oct 31 01:33:14.880911 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:14.880885    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-79z4l_eee8d0cb-1686-4b84-ac7c-e07f8612d4ba/kube-rbac-proxy-self/0.log"
Oct 31 01:33:14.933445 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:14.933417    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6db4c97dcc-87jqk_c1ada52a-5abd-4603-922d-cb20665f8a18/metrics-server/0.log"
Oct 31 01:33:14.963246 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:14.963201    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-sddw2_9789f7f7-7d7d-4cd1-b74e-fd960dbb0283/monitoring-plugin/0.log"
Oct 31 01:33:15.149936 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.149904    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z9b9p_02e925ec-3898-42da-a8c2-5a1998d17c48/node-exporter/0.log"
Oct 31 01:33:15.200455 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.200425    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z9b9p_02e925ec-3898-42da-a8c2-5a1998d17c48/kube-rbac-proxy/0.log"
Oct 31 01:33:15.224739 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.224711    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z9b9p_02e925ec-3898-42da-a8c2-5a1998d17c48/init-textfile/0.log"
Oct 31 01:33:15.258243 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.258197    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rxckd_ba2e347b-434a-437a-bf36-85a85efd81bd/kube-rbac-proxy-main/0.log"
Oct 31 01:33:15.283422 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.283390    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rxckd_ba2e347b-434a-437a-bf36-85a85efd81bd/kube-rbac-proxy-self/0.log"
Oct 31 01:33:15.309013 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.308955    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rxckd_ba2e347b-434a-437a-bf36-85a85efd81bd/openshift-state-metrics/0.log"
Oct 31 01:33:15.371085 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.371049    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/prometheus/0.log"
Oct 31 01:33:15.393450 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.393420    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/config-reloader/0.log"
Oct 31 01:33:15.420567 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.420486    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/thanos-sidecar/0.log"
Oct 31 01:33:15.450034 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.450003    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/kube-rbac-proxy-web/0.log"
Oct 31 01:33:15.475385 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.475337    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/kube-rbac-proxy/0.log"
Oct 31 01:33:15.503301 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.503258    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/kube-rbac-proxy-thanos/0.log"
Oct 31 01:33:15.530960 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.530929    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04279f09-aeff-4760-9418-c5f878b36e46/init-config-reloader/0.log"
Oct 31 01:33:15.572948 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.572911    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-dzqr7_ed848351-036d-461e-9295-e8eb410759cc/prometheus-operator/0.log"
Oct 31 01:33:15.594322 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.594294    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-dzqr7_ed848351-036d-461e-9295-e8eb410759cc/kube-rbac-proxy/0.log"
Oct 31 01:33:15.629856 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.629824    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-mqpr5_ab0a4936-76a5-40eb-a7bd-0822ea7c8de9/prometheus-operator-admission-webhook/0.log"
Oct 31 01:33:15.664844 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.664812    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68b5c76845-8wplb_d6f46adb-16dc-4058-bfd1-d33fca2885a3/telemeter-client/0.log"
Oct 31 01:33:15.693966 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.693889    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68b5c76845-8wplb_d6f46adb-16dc-4058-bfd1-d33fca2885a3/reload/0.log"
Oct 31 01:33:15.717553 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.713062    8029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"]
Oct 31 01:33:15.731901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.730421    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.731901 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.731828    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"]
Oct 31 01:33:15.737546 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.737517    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68b5c76845-8wplb_d6f46adb-16dc-4058-bfd1-d33fca2885a3/kube-rbac-proxy/0.log"
Oct 31 01:33:15.758526 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.758486    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-proc\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.758872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.758676    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-podres\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.758872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.758789    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-lib-modules\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.758872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.758821    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-sys\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.758872 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.758853    8029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xghx\" (UniqueName: \"kubernetes.io/projected/5acfc59b-138f-4016-9bfb-88a1c379907b-kube-api-access-9xghx\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.781307 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.781267    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7fd74876-fxzt7_9af1e2d1-ebb9-4941-8b61-dd640def4b9d/thanos-query/0.log"
Oct 31 01:33:15.817517 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.817484    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7fd74876-fxzt7_9af1e2d1-ebb9-4941-8b61-dd640def4b9d/kube-rbac-proxy-web/0.log"
Oct 31 01:33:15.847434 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.847403    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7fd74876-fxzt7_9af1e2d1-ebb9-4941-8b61-dd640def4b9d/kube-rbac-proxy/0.log"
Oct 31 01:33:15.860531 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860497    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-proc\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860650    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-proc\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860658    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-podres\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860772 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860719    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-podres\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860771    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-lib-modules\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860797    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-sys\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860817    8029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xghx\" (UniqueName: \"kubernetes.io/projected/5acfc59b-138f-4016-9bfb-88a1c379907b-kube-api-access-9xghx\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.860951 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860889    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-lib-modules\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.861111 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.860948    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5acfc59b-138f-4016-9bfb-88a1c379907b-sys\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.869048 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.869015    8029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xghx\" (UniqueName: \"kubernetes.io/projected/5acfc59b-138f-4016-9bfb-88a1c379907b-kube-api-access-9xghx\") pod \"perf-node-gather-daemonset-4rxmh\" (UID: \"5acfc59b-138f-4016-9bfb-88a1c379907b\") " pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:15.871950 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.871925    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7fd74876-fxzt7_9af1e2d1-ebb9-4941-8b61-dd640def4b9d/prom-label-proxy/0.log"
Oct 31 01:33:15.898493 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.898430    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7fd74876-fxzt7_9af1e2d1-ebb9-4941-8b61-dd640def4b9d/kube-rbac-proxy-rules/0.log"
Oct 31 01:33:15.929467 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:15.929291    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7fd74876-fxzt7_9af1e2d1-ebb9-4941-8b61-dd640def4b9d/kube-rbac-proxy-metrics/0.log"
Oct 31 01:33:16.057755 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:16.057660    8029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:16.358079 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:16.356545    8029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"]
Oct 31 01:33:16.365626 ip-10-0-140-244 kubenswrapper[8029]: W1031 01:33:16.363096    8029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5acfc59b_138f_4016_9bfb_88a1c379907b.slice/crio-2b3e6ec2ec02bf41181204a44441d40785b6bb479068881f92746e3db56b1a52 WatchSource:0}: Error finding container 2b3e6ec2ec02bf41181204a44441d40785b6bb479068881f92746e3db56b1a52: Status 404 returned error can't find the container with id 2b3e6ec2ec02bf41181204a44441d40785b6bb479068881f92746e3db56b1a52
Oct 31 01:33:17.109327 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:17.109278    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh" event={"ID":"5acfc59b-138f-4016-9bfb-88a1c379907b","Type":"ContainerStarted","Data":"34779b5ce1076f3515a5e7723a4f4974e583bff1fe3142b4ed0995b3fc578723"}
Oct 31 01:33:17.109917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:17.109354    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:17.109917 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:17.109374    8029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh" event={"ID":"5acfc59b-138f-4016-9bfb-88a1c379907b","Type":"ContainerStarted","Data":"2b3e6ec2ec02bf41181204a44441d40785b6bb479068881f92746e3db56b1a52"}
Oct 31 01:33:17.128849 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:17.128776    8029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh" podStartSLOduration=2.128753984 podStartE2EDuration="2.128753984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 01:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 01:33:17.127313065 +0000 UTC m=+2194.114425046" watchObservedRunningTime="2026-10-31 01:33:17.128753984 +0000 UTC m=+2194.115865961"
Oct 31 01:33:18.389391 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:18.389346    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6599b76c78-26thf_9c0051e9-8149-4f06-a8c6-147bd62d8436/console/0.log"
Oct 31 01:33:19.460536 ip-10-0-140-244 kubenswrapper[8029]: E1031 01:33:19.460234    8029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p2bzn" containerName="extractor"
Oct 31 01:33:20.392869 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:20.392825    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p9js6_e575cbe0-30bf-4286-a382-ffe481a27c91/dns/0.log"
Oct 31 01:33:20.420474 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:20.420440    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p9js6_e575cbe0-30bf-4286-a382-ffe481a27c91/kube-rbac-proxy/0.log"
Oct 31 01:33:20.499498 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:20.499471    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddx8v_ea64b244-e85f-494e-a824-9eba8ccfe22c/dns-node-resolver/0.log"
Oct 31 01:33:21.206909 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:21.206877    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hjrz2_f5a21f30-84ee-4b17-93f6-8da1e8ecdf0a/node-ca/0.log"
Oct 31 01:33:23.129731 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:23.129698    8029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fsd8j/perf-node-gather-daemonset-4rxmh"
Oct 31 01:33:23.953125 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:23.953090    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-858778b8-88xjg_1e413599-cc6e-4805-85f4-c27406052466/kube-auth-proxy/0.log"
Oct 31 01:33:24.221206 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:24.221125    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-qlmqk_d6fd01e9-d8ad-4360-8648-e90ff85643b7/istio-proxy/0.log"
Oct 31 01:33:24.256411 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:24.256377    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-sds66_38590073-ac40-4f8e-8295-5bc08ebc313b/istio-proxy/0.log"
Oct 31 01:33:24.293563 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:24.293531    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-tw6hh_5bc97d91-1e29-430f-b333-61b323418290/istio-proxy/0.log"
Oct 31 01:33:26.210267 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:26.209754    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r9zb2_319e5395-cce3-4e4d-9c3c-111e8eb1baaa/serve-healthcheck-canary/0.log"
Oct 31 01:33:27.042879 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:27.042846    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p2bzn_4c482845-4e0d-441c-aa58-44672e7f6bf3/kube-rbac-proxy/0.log"
Oct 31 01:33:27.066199 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:27.066170    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p2bzn_4c482845-4e0d-441c-aa58-44672e7f6bf3/exporter/0.log"
Oct 31 01:33:27.098056 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:27.098023    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p2bzn_4c482845-4e0d-441c-aa58-44672e7f6bf3/extractor/0.log"
Oct 31 01:33:28.578915 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:28.578884    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-858778b8-88xjg_1e413599-cc6e-4805-85f4-c27406052466/kube-auth-proxy/0.log"
Oct 31 01:33:28.705774 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:28.705713    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-qlmqk_d6fd01e9-d8ad-4360-8648-e90ff85643b7/istio-proxy/0.log"
Oct 31 01:33:28.724305 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:28.724268    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-sds66_38590073-ac40-4f8e-8295-5bc08ebc313b/istio-proxy/0.log"
Oct 31 01:33:28.743406 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:28.743372    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-tw6hh_5bc97d91-1e29-430f-b333-61b323418290/istio-proxy/0.log"
Oct 31 01:33:29.768246 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:29.768214    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-7f58f8c74c-5b8p5_6a98def7-65a7-4798-b8d6-73111762fe09/manager/0.log"
Oct 31 01:33:29.793809 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:29.793776    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-7f58f8c74c-5b8p5_6a98def7-65a7-4798-b8d6-73111762fe09/copy-manifests/0.log"
Oct 31 01:33:31.258171 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:31.258132    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-5d466cf57f-6xz5v_29aaf902-59b6-41f3-9f3e-2af1e9e9fec6/manager/0.log"
Oct 31 01:33:31.303625 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:31.303004    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-hdssj_6be48609-5359-44ac-b7f5-d8872cbdec93/jobset-operator/0.log"
Oct 31 01:33:32.385470 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:32.385410    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-858778b8-88xjg_1e413599-cc6e-4805-85f4-c27406052466/kube-auth-proxy/0.log"
Oct 31 01:33:32.571881 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:32.571844    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-qlmqk_d6fd01e9-d8ad-4360-8648-e90ff85643b7/istio-proxy/0.log"
Oct 31 01:33:32.602189 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:32.602152    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-sds66_38590073-ac40-4f8e-8295-5bc08ebc313b/istio-proxy/0.log"
Oct 31 01:33:32.621091 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:32.621057    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-tw6hh_5bc97d91-1e29-430f-b333-61b323418290/istio-proxy/0.log"
Oct 31 01:33:32.625656 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:32.625629    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmb5t_65e086a0-1eb5-46cf-895e-8a9cd892ce6b/server/0.log"
Oct 31 01:33:32.955782 ip-10-0-140-244 kubenswrapper[8029]: I1031 01:33:32.955745    8029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-nmdfz_3c50d69b-8557-482a-8329-976d4556bf95/manager/0.log"