Sep 18 18:42:22.991565 ip-10-0-132-94 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 18:42:22.991579 ip-10-0-132-94 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 18:42:22.991588 ip-10-0-132-94 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 18:42:22.991794 ip-10-0-132-94 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 18:42:33.164237 ip-10-0-132-94 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 18:42:33.164253 ip-10-0-132-94 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 94a0c6fabbfd4ba987c59a0b845f854a --
Sep 18 18:44:28.965889 ip-10-0-132-94 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 18:44:29.368147 ip-10-0-132-94 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:44:29.368147 ip-10-0-132-94 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 18:44:29.368147 ip-10-0-132-94 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:44:29.368147 ip-10-0-132-94 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 18:44:29.368788 ip-10-0-132-94 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:44:29.368788 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.368219    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 18:44:29.371102 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371087    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:29.371102 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371103    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371107    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371110    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371115    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371119    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371122    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371125    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371128    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371131    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371134    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371136    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371139    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371146    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371149    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371152    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371154    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371157    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371160    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371164    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:29.371167 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371167    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371170    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371173    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371176    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371179    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371182    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371185    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371187    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371190    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371192    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371195    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371197    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371200    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371202    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371205    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371207    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371210    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371212    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371215    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371218    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:29.371605 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371220    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371223    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371225    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371229    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371232    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371234    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371237    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371240    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371242    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371245    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371247    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371250    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371253    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371255    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371259    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371261    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371264    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371267    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371270    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:29.372115 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371272    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371275    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371277    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371280    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371283    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371286    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371290    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371293    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371295    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371298    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371300    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371302    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371305    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371308    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371312    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371314    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371317    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371320    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371327    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371330    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:29.372599 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371333    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371335    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371338    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371340    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371343    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371346    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371349    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371352    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371354    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371357    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371360    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371362    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371365    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371367    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371370    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371373    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371375    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371378    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371380    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.371383    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:29.373101 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372695    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372704    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372707    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372710    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372713    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372717    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372719    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372722    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372725    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372727    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372730    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372733    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372735    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372738    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372741    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372744    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372746    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372749    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372752    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372755    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:29.373611 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372757    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372760    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372763    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372765    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372768    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372771    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372774    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372777    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372779    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372782    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372785    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372787    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372790    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372793    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372795    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372798    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372800    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372803    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372807    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372812    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:29.374121 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372815    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372817    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372820    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372823    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372827    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372831    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372835    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372837    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372840    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372843    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372846    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372849    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372851    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372854    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372857    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372860    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372864    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372867    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372870    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:29.374679 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372872    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372875    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372877    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372880    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372882    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372885    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372887    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372890    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372892    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372895    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372898    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372901    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372907    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372910    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372913    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372916    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372919    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372921    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372924    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:29.375139 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372926    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372929    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372932    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372935    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372938    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372941    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372943    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372946    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372948    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372951    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372954    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372956    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372959    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372962    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372964    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372967    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372970    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372972    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372975    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372978    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:29.375597 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.372980    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373385    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373391    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373395    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373398    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373402    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373406    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373409    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373412    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373415    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373418    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373422    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373426    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373430    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373433    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373437    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373440    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373443    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373446    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:29.376090 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373448    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373451    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373454    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373456    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373459    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373461    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373464    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373467    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373469    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373474    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373477    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373480    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373483    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373486    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373489    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373492    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373494    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373497    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373499    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:29.376547 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373502    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373505    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373507    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373510    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373512    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373515    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373518    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373520    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373523    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373525    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373528    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373531    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373533    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373536    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373538    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373541    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373543    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373546    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373549    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373551    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:29.377018 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373554    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373556    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373559    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373561    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373564    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373566    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373569    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373571    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373574    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373576    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373578    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373581    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373583    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373586    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373589    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373591    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373594    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373598    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373601    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373603    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:29.377505 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373606    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373609    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373611    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373614    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373616    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373619    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373622    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373637    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373640    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373643    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373645    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373647    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373650    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373653    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373655    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373657    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373660    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373662    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373665    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373667    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:29.377992 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373672    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.373675    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373735    2568 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373768    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373777    2568 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373781    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373786    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373789    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373793    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373798    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373801    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373804    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373808    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373811    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373814    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373817    2568 flags.go:64] FLAG: --cgroup-root=""
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373820    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373823    2568 flags.go:64] FLAG: --client-ca-file=""
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373827    2568 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373829    2568 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373834    2568 flags.go:64] FLAG: --cluster-domain=""
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373837    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373840    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373844    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 18:44:29.378462 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373847    2568 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373850    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373853    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373856    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373860    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373863    2568 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373866    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373869    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373873    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373876    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373880    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373883    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373886    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373889    2568 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373892    2568 flags.go:64] FLAG: --enable-server="true"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373896    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373900    2568 flags.go:64] FLAG: --event-burst="100"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373903    2568 flags.go:64] FLAG: --event-qps="50"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373906    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373909    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373912    2568 flags.go:64] FLAG: --eviction-hard=""
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373916    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373919    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373922    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373925    2568 flags.go:64] FLAG: --eviction-soft=""
Sep 18 18:44:29.379052 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373928    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373930    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373933    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373936    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373939    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373942    2568 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373944    2568 flags.go:64] FLAG: --feature-gates=""
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373948    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373951    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373954    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373957    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373960    2568 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373963    2568 flags.go:64] FLAG: --help="false"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373966    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373969    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373973    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373976    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373980    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373983    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373986    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373989    2568 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373992    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373994    2568 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 18:44:29.379682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.373998    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374001    2568 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374004    2568 flags.go:64] FLAG: --kube-reserved=""
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374007    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374009    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374013    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374016    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374018    2568 flags.go:64] FLAG: --lock-file=""
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374021    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374024    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374027    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374032    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374035    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374037    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374040    2568 flags.go:64] FLAG: --logging-format="text"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374043    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374046    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374049    2568 flags.go:64] FLAG: --manifest-url=""
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374052    2568 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374056    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374059    2568 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374062    2568 flags.go:64] FLAG: --max-pods="110"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374065    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374068    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374073    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 18:44:29.380276 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374076    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374079    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374081    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374084    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374091    2568 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374094    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374097    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374100    2568 flags.go:64] FLAG: --pod-cidr=""
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374105    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374110    2568 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374113    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374116    2568 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374119    2568 flags.go:64] FLAG: --port="10250"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374122    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374126    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0faa0d9803b845cbf"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374129    2568 flags.go:64] FLAG: --qos-reserved=""
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374132    2568 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374135    2568 flags.go:64] FLAG: --register-node="true"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374138    2568 flags.go:64] FLAG: --register-with-taints=""
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374141    2568 flags.go:64] FLAG: --registry-burst="10"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374145    2568 flags.go:64] FLAG: --registry-qps="5"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374147    2568 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374150    2568 flags.go:64] FLAG: --reserved-memory=""
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374154    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374157    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 18:44:29.380907 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374160    2568 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374163    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374165    2568 flags.go:64] FLAG: --runonce="false"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374168    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374171    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374174    2568 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374178    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374181    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374184    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374187    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374189    2568 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374192    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374195    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374198    2568 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374201    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374204    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374209    2568 flags.go:64] FLAG: --system-cgroups=""
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374212    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374218    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374221    2568 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374223    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374228    2568 flags.go:64] FLAG: --tls-min-version=""
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374230    2568 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374233    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374236    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 18:44:29.381520 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374239    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374242    2568 flags.go:64] FLAG: --v="2"
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374246    2568 flags.go:64] FLAG: --version="false"
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374250    2568 flags.go:64] FLAG: --vmodule=""
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374253    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374256    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374383    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374387    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374390    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374393    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374395    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374398    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374401    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374405    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374408    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374411    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374413    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374416    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374419    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374422    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374424    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:29.382152 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374427    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374429    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374433    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374441    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374444    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374447    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374449    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374452    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374455    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374457    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374460    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374463    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374465    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374468    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374470    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374473    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374475    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374478    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374481    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374483    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374486    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:29.382686 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374488    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374492    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374495    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374500    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374503    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374505    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374508    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374511    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374513    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374516    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374518    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374521    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374524    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374529    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374532    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374536    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374539    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374542    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374545    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:29.383189 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374549    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374552    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374554    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374557    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374560    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374562    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374565    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374567    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374570    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374573    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374575    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374578    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374580    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374585    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374588    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374591    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374596    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374598    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374601    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:29.383669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374603    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374606    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374609    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374611    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374614    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374617    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374620    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374638    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374642    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374646    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374651    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374655    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374658    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374660    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374663    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374666    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374669    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374671    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374674    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:29.384140 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374677    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374680    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374682    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374685    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374688    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.374691    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.374696    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.382051    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.382067    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.382703    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.382713    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382775    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382780    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382784    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:29.384669 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382787    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382790    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382793    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382796    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382799    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382802    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382804    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382807    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382810    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382812    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382815    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382817    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382820    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382822    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382825    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382828    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382831    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382833    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382836    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:29.385029 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382839    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382841    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382844    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382846    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382849    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382851    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382854    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382865    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382868    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382871    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382873    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382877    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382881    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382884    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382887    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382890    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382892    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382895    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382897    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:29.385737 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382900    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382903    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382905    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382908    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382910    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382913    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382916    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382918    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382921    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382923    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382926    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382928    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382931    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382934    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382936    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382939    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382942    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382944    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382947    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382949    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:29.386319 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382953    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382961    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382964    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382967    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382969    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382972    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382974    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382977    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382979    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382982    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382985    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382988    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382990    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382993    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382995    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.382997    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383000    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383003    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383005    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:29.386834 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383007    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383012    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383015    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383018    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383021    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383024    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383026    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383029    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383031    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383034    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383036    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383039    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383042    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383045    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383047    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383055    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383058    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383061    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:29.387326 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383065    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.383070    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383215    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383220    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383223    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383227    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383229    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383232    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383235    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383238    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383241    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383244    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383246    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383249    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383252    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:44:29.387925 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383255    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383259    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383261    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383264    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383266    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383269    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383272    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383274    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383277    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383280    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383282    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383285    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383287    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383290    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383292    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383300    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383303    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383306    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383308    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:44:29.388298 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383311    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383314    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383318    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383321    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383324    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383327    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383330    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383333    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383336    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383339    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383342    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383346    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383349    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383352    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383355    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383358    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383361    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383364    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383366    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:44:29.388803 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383369    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383371    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383374    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383376    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383379    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383381    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383384    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383386    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383389    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383391    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383399    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383402    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383405    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383407    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383411    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383414    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383416    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383419    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383421    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:44:29.389266 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383424    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383426    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383429    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383431    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383434    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383436    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383439    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383441    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383444    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383446    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383449    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383452    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383454    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383456    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383459    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383462    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383464    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383467    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383469    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383472    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:44:29.389847 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383475    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383477    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383480    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383482    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383490    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383493    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383495    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383498    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:29.383500    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.383505    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.384435    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.388572    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.389482    2568 server.go:1019] "Starting client certificate rotation"
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.389585    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:44:29.390312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.389617    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:44:29.391873 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.391860    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:44:29.393109 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.393090    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:44:29.402372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.402356    2568 log.go:25] "Validated CRI v1 runtime API"
Sep 18 18:44:29.403915 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.403902    2568 log.go:25] "Validated CRI v1 image API"
Sep 18 18:44:29.404795 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.404779    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 18:44:29.407864 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.407846    2568 fs.go:135] Filesystem UUIDs: map[639ad07d-d624-4289-b7d6-86b0946ca59b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b1befe5f-125b-4f5a-9f3f-bef4488949e6:/dev/nvme0n1p3]
Sep 18 18:44:29.407917 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.407864    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 18:44:29.412754 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.412644    2568 manager.go:216] Machine: {Timestamp:2026-09-18 18:44:29.412140745 +0000 UTC m=+0.328287405 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103334 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec254c5b4be897544529cea16c5bcb7e SystemUUID:ec254c5b-4be8-9754-4529-cea16c5bcb7e BootID:94a0c6fa-bbfd-4ba9-87c5-9a0b845f854a Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e5:95:75:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e5:95:75:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:a2:c1:ae:0a:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 18:44:29.412754 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.412752    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 18:44:29.412870 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.412829    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 18:44:29.416000 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.415969    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 18:44:29.416485 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.416005    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 18:44:29.417057 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.417047    2568 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 18:44:29.417090 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.417061    2568 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 18:44:29.417090 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.417069    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:44:29.417799 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.417782    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:44:29.418525 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.418514    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:44:29.418600 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.418592    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 18:44:29.421549 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421538    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:44:29.421682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421672    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 18:44:29.421713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421702    2568 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 18:44:29.421713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421711    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 18:44:29.421781 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421723    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 18:44:29.421781 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421732    2568 kubelet.go:398] "Adding apiserver pod source"
Sep 18 18:44:29.421781 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.421740    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 18:44:29.422066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.422052    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:44:29.422129 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.422075    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:44:29.424961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.424943    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 18:44:29.427349 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.427336    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 18:44:29.427409 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.427366    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 18:44:29.429079 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429066    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429086    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429092    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429098    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429106    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429112    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429126    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 18:44:29.429132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429131    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 18:44:29.429327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429140    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 18:44:29.429327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429147    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 18:44:29.429327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429155    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 18:44:29.429327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429163    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 18:44:29.429921 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.429902    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 18:44:29.429964 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429959    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 18:44:29.429997 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.429968    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 18:44:29.430112 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.430094    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 18:44:29.433761 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.433739    2568 server.go:1268] "Started kubelet"
Sep 18 18:44:29.433901 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.433845    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 18:44:29.433972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.433924    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 18:44:29.434305 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.434280    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 18:44:29.434431 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.434412    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-94.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 18:44:29.434698 ip-10-0-132-94 systemd[1]: Started Kubernetes Kubelet.
Sep 18 18:44:29.434898 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.434864    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 18:44:29.436288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.436273    2568 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 18:44:29.439498 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.439474    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvd2b"
Sep 18 18:44:29.440670 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.440652    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 18:44:29.440818 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.440786    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 18:44:29.441101 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.441088    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 18:44:29.441181 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.440300    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-94.ec2.internal.18d67e9275d72987  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-94.ec2.internal,UID:ip-10-0-132-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-94.ec2.internal,},FirstTimestamp:2026-09-18 18:44:29.433710983 +0000 UTC m=+0.349857642,LastTimestamp:2026-09-18 18:44:29.433710983 +0000 UTC m=+0.349857642,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-94.ec2.internal,}"
Sep 18 18:44:29.441262 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.441202    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:29.441716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.441698    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 18:44:29.441716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.441698    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 18:44:29.441842 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.441722    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 18:44:29.441842 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.441790    2568 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 18:44:29.441842 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.441802    2568 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 18:44:29.442265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442247    2568 factory.go:55] Registering systemd factory
Sep 18 18:44:29.442330 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442269    2568 factory.go:223] Registration of the systemd container factory successfully
Sep 18 18:44:29.442539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442527    2568 factory.go:153] Registering CRI-O factory
Sep 18 18:44:29.442591 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442542    2568 factory.go:223] Registration of the crio container factory successfully
Sep 18 18:44:29.442652 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442596    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 18:44:29.442652 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442615    2568 factory.go:103] Registering Raw factory
Sep 18 18:44:29.442652 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.442648    2568 manager.go:1196] Started watching for new ooms in manager
Sep 18 18:44:29.443038 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.443024    2568 manager.go:349] Starting recovery of all containers
Sep 18 18:44:29.446253 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.446213    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 18:44:29.446364 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.446342    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 18:44:29.448488 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.448466    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvd2b"
Sep 18 18:44:29.453838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.453550    2568 manager.go:354] Recovery completed
Sep 18 18:44:29.458862 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.458849    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.460463 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.460444    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.460563 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.460474    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.460563 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.460483    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.461133 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461116    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.461133 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461129    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 18:44:29.461271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461140    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 18:44:29.461271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461156    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:44:29.461928 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461912    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.461981 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461943    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.461981 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.461957    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.462310 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.462300    2568 policy_none.go:49] "None policy: Start"
Sep 18 18:44:29.462347 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.462315    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 18:44:29.462347 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.462324    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 18:44:29.462847 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.462774    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-94.ec2.internal.18d67e92776f5566  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-94.ec2.internal,UID:ip-10-0-132-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-94.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-94.ec2.internal,},FirstTimestamp:2026-09-18 18:44:29.460460902 +0000 UTC m=+0.376607561,LastTimestamp:2026-09-18 18:44:29.460460902 +0000 UTC m=+0.376607561,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-94.ec2.internal,}"
Sep 18 18:44:29.463298 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.463287    2568 policy_none.go:47] "Start"
Sep 18 18:44:29.485247 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485189    2568 manager.go:337] "Starting Device Plugin manager"
Sep 18 18:44:29.485247 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.485226    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 18:44:29.485247 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485238    2568 server.go:87] "Starting device plugin registration server"
Sep 18 18:44:29.485570 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485554    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 18:44:29.485677 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485570    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 18:44:29.485732 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485724    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 18:44:29.485803 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485789    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 18:44:29.485857 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.485804    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 18:44:29.486304 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.486283    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 18:44:29.486391 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.486331    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:29.568770 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.568731    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 18:44:29.569888 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.569872    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 18:44:29.569961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.569915    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 18:44:29.569961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.569936    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 18:44:29.570062 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.569971    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 18:44:29.572567 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.572549    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:29.585668 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.585638    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.587602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.587584    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.587690 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.587619    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.587690 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.587647    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.587690 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.587673    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.595961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.595947    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.596046 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.595970    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-94.ec2.internal\": node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:29.610839 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.610815    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:29.670771 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.670751    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"]
Sep 18 18:44:29.670840 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.670805    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.672145 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.672121    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.672233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.672149    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.672233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.672159    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.673228 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673217    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.673349 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673335    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.673387 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673364    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.673919 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673901    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.674009 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673932    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.674009 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673946    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.674009 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.673904    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.674009 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.674008    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.674193 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.674031    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.675190 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.675175    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.675269 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.675205    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:44:29.675864 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.675850    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:44:29.675922 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.675875    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:44:29.675922 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.675887    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:44:29.699216 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.699195    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-94.ec2.internal\" not found" node="ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.703528 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.703513    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-94.ec2.internal\" not found" node="ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.711003 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.710988    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:29.743311 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.743291    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d97b59528ec8f7f1a82a1af8817dbcf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal\" (UID: \"d97b59528ec8f7f1a82a1af8817dbcf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.743383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.743315    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d97b59528ec8f7f1a82a1af8817dbcf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal\" (UID: \"d97b59528ec8f7f1a82a1af8817dbcf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.743383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.743331    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8708fb89a1341e781b6f6cd3d321ee06-config\") pod \"kube-apiserver-proxy-ip-10-0-132-94.ec2.internal\" (UID: \"8708fb89a1341e781b6f6cd3d321ee06\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.811915 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.811895    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:29.843770 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.843752    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d97b59528ec8f7f1a82a1af8817dbcf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal\" (UID: \"d97b59528ec8f7f1a82a1af8817dbcf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.843837 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.843773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d97b59528ec8f7f1a82a1af8817dbcf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal\" (UID: \"d97b59528ec8f7f1a82a1af8817dbcf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.843837 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.843791    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8708fb89a1341e781b6f6cd3d321ee06-config\") pod \"kube-apiserver-proxy-ip-10-0-132-94.ec2.internal\" (UID: \"8708fb89a1341e781b6f6cd3d321ee06\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.843837 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.843816    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8708fb89a1341e781b6f6cd3d321ee06-config\") pod \"kube-apiserver-proxy-ip-10-0-132-94.ec2.internal\" (UID: \"8708fb89a1341e781b6f6cd3d321ee06\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.843960 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.843853    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d97b59528ec8f7f1a82a1af8817dbcf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal\" (UID: \"d97b59528ec8f7f1a82a1af8817dbcf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.843960 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:29.843857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d97b59528ec8f7f1a82a1af8817dbcf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal\" (UID: \"d97b59528ec8f7f1a82a1af8817dbcf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:29.911995 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:29.911959    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:30.002648 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:30.002560    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:30.006896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:30.006835    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:30.012050 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:30.012030    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:30.112566 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:30.112527    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:31.690946 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:31.690924    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:31.783476 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.783406    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:31.791744 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:31.791722    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:31.868200 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.868175    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 18:44:31.868632 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.868325    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:44:31.868632 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.868334    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:44:31.892496 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:31.892470    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:31.935498 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.935466    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 18:39:30 +0000 UTC" deadline="2028-06-05 14:09:00.341282208 +0000 UTC"
Sep 18 18:44:31.935498 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.935492    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15019h24m28.405793018s"
Sep 18 18:44:31.949185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:31.949167    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:31.992990 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:31.992956    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:32.093462 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:32.093430    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:32.100726 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:32.100683    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8708fb89a1341e781b6f6cd3d321ee06.slice/crio-2f50d3e28772c9a12c7a3447b612da2a35c8423067f67226f621ce3af08d51e4 WatchSource:0}: Error finding container 2f50d3e28772c9a12c7a3447b612da2a35c8423067f67226f621ce3af08d51e4: Status 404 returned error can't find the container with id 2f50d3e28772c9a12c7a3447b612da2a35c8423067f67226f621ce3af08d51e4
Sep 18 18:44:32.100952 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:32.100929    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd97b59528ec8f7f1a82a1af8817dbcf5.slice/crio-cbc7d91b799e05dc75c32160d91e7ce670c816ca525e33cf5ce74f3ae2da11f4 WatchSource:0}: Error finding container cbc7d91b799e05dc75c32160d91e7ce670c816ca525e33cf5ce74f3ae2da11f4: Status 404 returned error can't find the container with id cbc7d91b799e05dc75c32160d91e7ce670c816ca525e33cf5ce74f3ae2da11f4
Sep 18 18:44:32.105335 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.105316    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:44:32.194547 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:32.194494    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-94.ec2.internal\" not found"
Sep 18 18:44:32.283833 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.283813    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:32.319621 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.319604    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:32.331284 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.331267    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:44:32.332217 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.332205    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal"
Sep 18 18:44:32.342643 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.342595    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:44:32.856802 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.856763    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:32.900701 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.900661    2568 apiserver.go:52] "Watching apiserver"
Sep 18 18:44:32.910282 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.910244    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 18:44:32.910803 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.910777    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-pqmtv","openshift-dns/node-resolver-nsfwm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal","openshift-multus/multus-77jzc","openshift-network-operator/iptables-alerter-96x66","openshift-ovn-kubernetes/ovnkube-node-85dvq","kube-system/global-pull-secret-syncer-hq4lf","kube-system/konnectivity-agent-nfq7b","kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal","openshift-image-registry/node-ca-vdgtt","openshift-multus/multus-additional-cni-plugins-k5z4l","openshift-multus/network-metrics-daemon-k77tr","openshift-network-diagnostics/network-check-target-k5248","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"]
Sep 18 18:44:32.913400 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.913269    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:32.913400 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:32.913355    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:32.915614 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.915502    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:32.915614 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:32.915595    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:32.917725 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.917700    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.918835 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.918812    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 18:44:32.920271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.920247    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.920460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.920445    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.920649 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.920632    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 18:44:32.920874 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.920857    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jrxx7\""
Sep 18 18:44:32.921378 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.921357    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.923483 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.923310    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.923598 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.923541    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wfw6r\""
Sep 18 18:44:32.923734 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.923717    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:32.923845 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.923827    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 18:44:32.923904 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.923827    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 18:44:32.924093 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.924075    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.926033 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.925736    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:32.926033 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.925926    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.926033 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.926004    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.926392 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.926371    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wfggf\""
Sep 18 18:44:32.926459 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.926392    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 18:44:32.928471 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.928452    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:32.929360 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.929340    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 18:44:32.929649 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.929630    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.929903 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.929888    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.930461 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930101    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 18:44:32.930461 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930271    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 18:44:32.930575 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930494    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bgjfn\""
Sep 18 18:44:32.930718 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930703    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 18:44:32.930900 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930877    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-device-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.930987 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-kubelet\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.930987 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930935    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-hostroot\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.930987 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930957    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-kubelet-config\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:32.930987 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.930981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.931202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931002    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk4tm\" (UniqueName: \"kubernetes.io/projected/6625f6df-5a26-4086-822a-a77557fb4057-kube-api-access-fk4tm\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.931202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931024    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44db16cc-8506-4ed0-b9af-78be01affad6-host-slash\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:32.931202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931077    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-registration-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.931202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931127    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 18:44:32.931202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931148    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-68xzp\""
Sep 18 18:44:32.931202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-conf-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931219    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgsgf\" (UniqueName: \"kubernetes.io/projected/3d4a16e1-bc0d-4298-8762-493d4d442b73-kube-api-access-rgsgf\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931266    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931291    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-cnibin\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931343    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-socket-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931361    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931363    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-system-cni-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931433    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-netns\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931470 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931458    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-cni-bin\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931484    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-daemon-config\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-multus-certs\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931601    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d4a16e1-bc0d-4298-8762-493d4d442b73-cni-binary-copy\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.931984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931970    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:32.932185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.931629    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-cni-multus\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.932185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932104    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-sys-fs\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.932185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-cni-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.932185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932150    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-socket-dir-parent\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.932185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44db16cc-8506-4ed0-b9af-78be01affad6-iptables-alerter-script\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:32.932419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932196    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-etc-selinux\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:32.932419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-k8s-cni-cncf-io\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.932419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932265    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kqf7\" (UniqueName: \"kubernetes.io/projected/44db16cc-8506-4ed0-b9af-78be01affad6-kube-api-access-8kqf7\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:32.932419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932287    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-etc-kubernetes\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:32.932419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932310    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-dbus\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:32.932679 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.932664    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:32.936700 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.936671    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:32.937144 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.937124    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:44:32.937333 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.937306    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vp66d\""
Sep 18 18:44:32.937438 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.937421    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.937503 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.937479    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.937725 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.937707    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.938461 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.938030    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.938698 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.938671    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dj59h\""
Sep 18 18:44:32.938972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.938953    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:32.939202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.939183    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 18:44:32.939768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.939748    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ctjn9\""
Sep 18 18:44:32.940031 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.940010    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 18:44:32.940416 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.940398    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 18:44:32.940972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.940951    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 18:44:32.941707 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.941687    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d2gf9\""
Sep 18 18:44:32.941916 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.941893    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 18:44:32.943094 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.943076    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:32.943220 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:32.943197    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:32.955289 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.955265    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nx84t"
Sep 18 18:44:32.963734 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:32.963707    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nx84t"
Sep 18 18:44:33.020864 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.020840    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 18:44:33.033425 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033397    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-cni-multus\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.033578 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033430    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-system-cni-dir\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.033578 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033457    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-log-socket\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.033578 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033474    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-modprobe-d\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.033578 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033538    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-cni-multus\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.033578 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-kubernetes\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033600    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-serviceca\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-cni-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033651    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-socket-dir-parent\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44db16cc-8506-4ed0-b9af-78be01affad6-iptables-alerter-script\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033707    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-env-overrides\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-ovnkube-script-lib\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-socket-dir-parent\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033751    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-cni-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.033838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-systemd\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033855    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-run\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033887    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-etc-selinux\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-cni-bin\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.033974    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-etc-selinux\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034000    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysctl-conf\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034024    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-etc-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-device-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-kubelet\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-hostroot\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034145    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-hostroot\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034145    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-kubelet\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034163    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysconfig\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034167    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-device-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034192    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8386b94-b143-4f08-a95a-ce6fb840c8f3-tmp-dir\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034216    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-node-log\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034227    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/44db16cc-8506-4ed0-b9af-78be01affad6-iptables-alerter-script\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.034254 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034240    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-run-ovn-kubernetes\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034291    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-kubelet\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034313    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-systemd-units\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034332    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034335    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-ovnkube-config\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3959247-ef44-4f78-9fd2-4f64325cc885-ovn-node-metrics-cert\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6pl2\" (UniqueName: \"kubernetes.io/projected/42dc14ed-ebaa-4847-91f3-acd381315150-kube-api-access-c6pl2\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034418    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-registration-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034447    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034481    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdx2v\" (UniqueName: \"kubernetes.io/projected/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-kube-api-access-zdx2v\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034494    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-registration-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034510    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034565    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-socket-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.034574    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034587    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-system-cni-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.035087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034612    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-cni-bin\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.034643    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:44:33.534624435 +0000 UTC m=+2.972884631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034663    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-var-lib-cni-bin\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034698    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-socket-dir\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034663    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-daemon-config\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034709    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-system-cni-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3e57d49-e5b9-454c-925c-24f923cf8c69-agent-certs\") pod \"konnectivity-agent-nfq7b\" (UID: \"d3e57d49-e5b9-454c-925c-24f923cf8c69\") " pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034783    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d4a16e1-bc0d-4298-8762-493d4d442b73-cni-binary-copy\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034842    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034864    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-slash\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4djs\" (UniqueName: \"kubernetes.io/projected/a3959247-ef44-4f78-9fd2-4f64325cc885-kube-api-access-s4djs\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034919    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42dc14ed-ebaa-4847-91f3-acd381315150-tmp\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034955    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-host\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.034983    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-sys-fs\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3e57d49-e5b9-454c-925c-24f923cf8c69-konnectivity-ca\") pod \"konnectivity-agent-nfq7b\" (UID: \"d3e57d49-e5b9-454c-925c-24f923cf8c69\") " pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.035773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-systemd\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6625f6df-5a26-4086-822a-a77557fb4057-sys-fs\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035066    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpwbn\" (UniqueName: \"kubernetes.io/projected/e8386b94-b143-4f08-a95a-ce6fb840c8f3-kube-api-access-gpwbn\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-k8s-cni-cncf-io\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035119    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kqf7\" (UniqueName: \"kubernetes.io/projected/44db16cc-8506-4ed0-b9af-78be01affad6-kube-api-access-8kqf7\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035153    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-cni-netd\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035183    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-daemon-config\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-sys\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035184    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-k8s-cni-cncf-io\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-var-lib-kubelet\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035243    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-etc-kubernetes\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-dbus\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d4a16e1-bc0d-4298-8762-493d4d442b73-cni-binary-copy\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035326    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-etc-kubernetes\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035364    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42dc14ed-ebaa-4847-91f3-acd381315150-etc-tuned\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035414    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-dbus\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:33.036355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035423    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-kubelet-config\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035451    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s288\" (UniqueName: \"kubernetes.io/projected/fe46cc04-7e73-4bcf-8a60-b684fb221935-kube-api-access-7s288\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035504    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035547    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-ovn\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035570    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysctl-d\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035589    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-host\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035510    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-kubelet-config\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e8386b94-b143-4f08-a95a-ce6fb840c8f3-hosts-file\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035647    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk4tm\" (UniqueName: \"kubernetes.io/projected/6625f6df-5a26-4086-822a-a77557fb4057-kube-api-access-fk4tm\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035671    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44db16cc-8506-4ed0-b9af-78be01affad6-host-slash\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035692    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cnibin\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-var-lib-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035730    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035730    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44db16cc-8506-4ed0-b9af-78be01affad6-host-slash\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ps7z\" (UniqueName: \"kubernetes.io/projected/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-kube-api-access-4ps7z\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035765    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-conf-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.036844 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035788    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgsgf\" (UniqueName: \"kubernetes.io/projected/3d4a16e1-bc0d-4298-8762-493d4d442b73-kube-api-access-rgsgf\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-run-netns\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035829    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-multus-conf-dir\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-lib-modules\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-cnibin\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035940    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.035961    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-netns\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.036002    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-multus-certs\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.036024    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-cnibin\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.036026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-netns\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.037327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.036091    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3d4a16e1-bc0d-4298-8762-493d4d442b73-host-run-multus-certs\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.042260 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.042220    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:33.042260 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.042243    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:33.042260 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.042256    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:33.042434 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.042317    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:44:33.542301011 +0000 UTC m=+2.980561219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:33.042434 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.042411    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 18:44:33.047087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.046668    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgsgf\" (UniqueName: \"kubernetes.io/projected/3d4a16e1-bc0d-4298-8762-493d4d442b73-kube-api-access-rgsgf\") pod \"multus-77jzc\" (UID: \"3d4a16e1-bc0d-4298-8762-493d4d442b73\") " pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.047087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.046747    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk4tm\" (UniqueName: \"kubernetes.io/projected/6625f6df-5a26-4086-822a-a77557fb4057-kube-api-access-fk4tm\") pod \"aws-ebs-csi-driver-node-m76np\" (UID: \"6625f6df-5a26-4086-822a-a77557fb4057\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.047087 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.047054    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kqf7\" (UniqueName: \"kubernetes.io/projected/44db16cc-8506-4ed0-b9af-78be01affad6-kube-api-access-8kqf7\") pod \"iptables-alerter-96x66\" (UID: \"44db16cc-8506-4ed0-b9af-78be01affad6\") " pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.053946 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.053887    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal" event={"ID":"d97b59528ec8f7f1a82a1af8817dbcf5","Type":"ContainerStarted","Data":"cbc7d91b799e05dc75c32160d91e7ce670c816ca525e33cf5ce74f3ae2da11f4"}
Sep 18 18:44:33.055424 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.055396    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal" event={"ID":"8708fb89a1341e781b6f6cd3d321ee06","Type":"ContainerStarted","Data":"2f50d3e28772c9a12c7a3447b612da2a35c8423067f67226f621ce3af08d51e4"}
Sep 18 18:44:33.136537 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-serviceca\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.136537 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136487    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-env-overrides\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136537 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136526    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-ovnkube-script-lib\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136551    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-systemd\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136573    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-run\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136596    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-cni-bin\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136620    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysctl-conf\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136644    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-etc-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysconfig\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136690    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8386b94-b143-4f08-a95a-ce6fb840c8f3-tmp-dir\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-node-log\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136735    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-run-ovn-kubernetes\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136758    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-kubelet\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.136791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136780    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-systemd-units\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136802    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-ovnkube-config\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3959247-ef44-4f78-9fd2-4f64325cc885-ovn-node-metrics-cert\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136861    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6pl2\" (UniqueName: \"kubernetes.io/projected/42dc14ed-ebaa-4847-91f3-acd381315150-kube-api-access-c6pl2\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136901    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdx2v\" (UniqueName: \"kubernetes.io/projected/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-kube-api-access-zdx2v\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.136978    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3e57d49-e5b9-454c-925c-24f923cf8c69-agent-certs\") pod \"konnectivity-agent-nfq7b\" (UID: \"d3e57d49-e5b9-454c-925c-24f923cf8c69\") " pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137002    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137052    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-slash\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137074    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4djs\" (UniqueName: \"kubernetes.io/projected/a3959247-ef44-4f78-9fd2-4f64325cc885-kube-api-access-s4djs\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137090    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-env-overrides\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137097    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42dc14ed-ebaa-4847-91f3-acd381315150-tmp\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137154    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-host\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137184    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3e57d49-e5b9-454c-925c-24f923cf8c69-konnectivity-ca\") pod \"konnectivity-agent-nfq7b\" (UID: \"d3e57d49-e5b9-454c-925c-24f923cf8c69\") " pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137214    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-systemd\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137238    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpwbn\" (UniqueName: \"kubernetes.io/projected/e8386b94-b143-4f08-a95a-ce6fb840c8f3-kube-api-access-gpwbn\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.137460 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137257    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-cni-netd\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137277    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-sys\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-var-lib-kubelet\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42dc14ed-ebaa-4847-91f3-acd381315150-etc-tuned\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137396    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s288\" (UniqueName: \"kubernetes.io/projected/fe46cc04-7e73-4bcf-8a60-b684fb221935-kube-api-access-7s288\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137444    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137471    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-ovn\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137495    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysctl-d\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137534    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-host\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137558    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e8386b94-b143-4f08-a95a-ce6fb840c8f3-hosts-file\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cnibin\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-var-lib-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137663    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ps7z\" (UniqueName: \"kubernetes.io/projected/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-kube-api-access-4ps7z\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137692    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-run-netns\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.138318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137716    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-lib-modules\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137763    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-system-cni-dir\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137767    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-ovnkube-script-lib\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-log-socket\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-modprobe-d\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137837    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-kubernetes\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-systemd\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.137936    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysconfig\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138115    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8386b94-b143-4f08-a95a-ce6fb840c8f3-tmp-dir\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138193    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-node-log\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138241    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-run-ovn-kubernetes\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138284    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-kubelet\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138312    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-serviceca\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138333    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-systemd-units\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138358    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-host\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138775    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3959247-ef44-4f78-9fd2-4f64325cc885-ovnkube-config\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138826    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3e57d49-e5b9-454c-925c-24f923cf8c69-konnectivity-ca\") pod \"konnectivity-agent-nfq7b\" (UID: \"d3e57d49-e5b9-454c-925c-24f923cf8c69\") " pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138867    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-slash\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.139265 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.138888    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-systemd\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139034    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-cni-netd\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-sys\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139117    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-system-cni-dir\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-kubernetes\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139214    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-lib-modules\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139230    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-run\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139260    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-cni-bin\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139261    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysctl-d\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139289    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-modprobe-d\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139297    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cnibin\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139315    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e8386b94-b143-4f08-a95a-ce6fb840c8f3-hosts-file\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139342    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-var-lib-kubelet\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139350    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139266    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-host\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139370    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42dc14ed-ebaa-4847-91f3-acd381315150-etc-sysctl-conf\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139381    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-log-socket\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139393    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-run-ovn\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.140179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139420    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-run-netns\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139435    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139738    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-var-lib-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139846    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3959247-ef44-4f78-9fd2-4f64325cc885-etc-openvswitch\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139892    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.139996    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.140063    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.140121    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:33.640102761 +0000 UTC m=+3.078362957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.140313    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.141070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.140712    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.141639 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.141617    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42dc14ed-ebaa-4847-91f3-acd381315150-tmp\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.142377 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.142355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3e57d49-e5b9-454c-925c-24f923cf8c69-agent-certs\") pod \"konnectivity-agent-nfq7b\" (UID: \"d3e57d49-e5b9-454c-925c-24f923cf8c69\") " pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.142819 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.142797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3959247-ef44-4f78-9fd2-4f64325cc885-ovn-node-metrics-cert\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.143433 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.143408    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42dc14ed-ebaa-4847-91f3-acd381315150-etc-tuned\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.147986 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.147971    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6pl2\" (UniqueName: \"kubernetes.io/projected/42dc14ed-ebaa-4847-91f3-acd381315150-kube-api-access-c6pl2\") pod \"tuned-pqmtv\" (UID: \"42dc14ed-ebaa-4847-91f3-acd381315150\") " pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.149121 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.149097    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ps7z\" (UniqueName: \"kubernetes.io/projected/1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4-kube-api-access-4ps7z\") pod \"node-ca-vdgtt\" (UID: \"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4\") " pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.150618 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.150591    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpwbn\" (UniqueName: \"kubernetes.io/projected/e8386b94-b143-4f08-a95a-ce6fb840c8f3-kube-api-access-gpwbn\") pod \"node-resolver-nsfwm\" (UID: \"e8386b94-b143-4f08-a95a-ce6fb840c8f3\") " pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.151169 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.151144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4djs\" (UniqueName: \"kubernetes.io/projected/a3959247-ef44-4f78-9fd2-4f64325cc885-kube-api-access-s4djs\") pod \"ovnkube-node-85dvq\" (UID: \"a3959247-ef44-4f78-9fd2-4f64325cc885\") " pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.151773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.151713    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdx2v\" (UniqueName: \"kubernetes.io/projected/71b2e4cb-ff67-4498-9222-ff7604a6d1bd-kube-api-access-zdx2v\") pod \"multus-additional-cni-plugins-k5z4l\" (UID: \"71b2e4cb-ff67-4498-9222-ff7604a6d1bd\") " pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.152183 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.152162    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s288\" (UniqueName: \"kubernetes.io/projected/fe46cc04-7e73-4bcf-8a60-b684fb221935-kube-api-access-7s288\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:33.231303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.231280    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np"
Sep 18 18:44:33.242524 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.242492    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-77jzc"
Sep 18 18:44:33.252104 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.252080    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-96x66"
Sep 18 18:44:33.258724 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.258703    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:33.266281 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.266264    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:33.273752 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.273735    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqmtv"
Sep 18 18:44:33.280330 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.280312    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nsfwm"
Sep 18 18:44:33.287930 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.287911    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vdgtt"
Sep 18 18:44:33.292465 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.292449    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k5z4l"
Sep 18 18:44:33.540605 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.540572    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:33.540765 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.540728    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:33.540815 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.540790    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:44:34.540773066 +0000 UTC m=+3.979033248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:33.641098 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.641065    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:33.641258 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.641129    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:33.641258 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.641220    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:33.641351 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.641286    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:34.641264099 +0000 UTC m=+4.079524281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:33.641351 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.641288    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:33.641351 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.641306    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:33.641351 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.641319    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:33.641531 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:33.641367    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:44:34.641352353 +0000 UTC m=+4.079612556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:33.966730 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.966681    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:39:32 +0000 UTC" deadline="2028-04-18 02:31:56.511103096 +0000 UTC"
Sep 18 18:44:33.966730 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:33.966722    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13855h47m22.544385087s"
Sep 18 18:44:34.044012 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.043711    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b2e4cb_ff67_4498_9222_ff7604a6d1bd.slice/crio-45bcf245a74690021fc3246df972d0328b6e920644abedbccba0762c76b748f0 WatchSource:0}: Error finding container 45bcf245a74690021fc3246df972d0328b6e920644abedbccba0762c76b748f0: Status 404 returned error can't find the container with id 45bcf245a74690021fc3246df972d0328b6e920644abedbccba0762c76b748f0
Sep 18 18:44:34.045550 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.045431    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4a16e1_bc0d_4298_8762_493d4d442b73.slice/crio-ace41ccaa303230cfc44fc7179b983516640a84bec94da943e1a7ff8b7e397e7 WatchSource:0}: Error finding container ace41ccaa303230cfc44fc7179b983516640a84bec94da943e1a7ff8b7e397e7: Status 404 returned error can't find the container with id ace41ccaa303230cfc44fc7179b983516640a84bec94da943e1a7ff8b7e397e7
Sep 18 18:44:34.048740 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.048715    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e57d49_e5b9_454c_925c_24f923cf8c69.slice/crio-6f931db82ac9b50890e61774d38d42fb5cd6209baddafb5ca0dcdb610769fc77 WatchSource:0}: Error finding container 6f931db82ac9b50890e61774d38d42fb5cd6209baddafb5ca0dcdb610769fc77: Status 404 returned error can't find the container with id 6f931db82ac9b50890e61774d38d42fb5cd6209baddafb5ca0dcdb610769fc77
Sep 18 18:44:34.049243 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.049219    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3959247_ef44_4f78_9fd2_4f64325cc885.slice/crio-b8cef0c55e7152a356169e9766ae9968c6e173b7ef5974b8af537b4f1195c00c WatchSource:0}: Error finding container b8cef0c55e7152a356169e9766ae9968c6e173b7ef5974b8af537b4f1195c00c: Status 404 returned error can't find the container with id b8cef0c55e7152a356169e9766ae9968c6e173b7ef5974b8af537b4f1195c00c
Sep 18 18:44:34.050173 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.050078    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8386b94_b143_4f08_a95a_ce6fb840c8f3.slice/crio-db3b3812fc7a537cb78dccbaf9d6722559eb018aa699c132eaa174afcdf79811 WatchSource:0}: Error finding container db3b3812fc7a537cb78dccbaf9d6722559eb018aa699c132eaa174afcdf79811: Status 404 returned error can't find the container with id db3b3812fc7a537cb78dccbaf9d6722559eb018aa699c132eaa174afcdf79811
Sep 18 18:44:34.050862 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.050761    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f03eb80_0bb5_4a2a_b2a4_a5cfc0431fd4.slice/crio-6a20f51232421ca096c0d522738713b85901f48b2fd42efde36038fb5420e3c5 WatchSource:0}: Error finding container 6a20f51232421ca096c0d522738713b85901f48b2fd42efde36038fb5420e3c5: Status 404 returned error can't find the container with id 6a20f51232421ca096c0d522738713b85901f48b2fd42efde36038fb5420e3c5
Sep 18 18:44:34.052565 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.052502    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6625f6df_5a26_4086_822a_a77557fb4057.slice/crio-12872c7fa493948db89569beaf95ca51739135087c722d4b85c1fb2d84835948 WatchSource:0}: Error finding container 12872c7fa493948db89569beaf95ca51739135087c722d4b85c1fb2d84835948: Status 404 returned error can't find the container with id 12872c7fa493948db89569beaf95ca51739135087c722d4b85c1fb2d84835948
Sep 18 18:44:34.054769 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.053887    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44db16cc_8506_4ed0_b9af_78be01affad6.slice/crio-221549c98d1b5ca6878d1264e90800a838b6b806081ea8c8d9c82785cb55bda3 WatchSource:0}: Error finding container 221549c98d1b5ca6878d1264e90800a838b6b806081ea8c8d9c82785cb55bda3: Status 404 returned error can't find the container with id 221549c98d1b5ca6878d1264e90800a838b6b806081ea8c8d9c82785cb55bda3
Sep 18 18:44:34.054769 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:44:34.054659    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42dc14ed_ebaa_4847_91f3_acd381315150.slice/crio-f0e2bdc9f6200e07593646ad6a60777179c48cf39bbb35dc7f113e61d5c6d488 WatchSource:0}: Error finding container f0e2bdc9f6200e07593646ad6a60777179c48cf39bbb35dc7f113e61d5c6d488: Status 404 returned error can't find the container with id f0e2bdc9f6200e07593646ad6a60777179c48cf39bbb35dc7f113e61d5c6d488
Sep 18 18:44:34.058510 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.058488    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-96x66" event={"ID":"44db16cc-8506-4ed0-b9af-78be01affad6","Type":"ContainerStarted","Data":"221549c98d1b5ca6878d1264e90800a838b6b806081ea8c8d9c82785cb55bda3"}
Sep 18 18:44:34.059840 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.059817    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np" event={"ID":"6625f6df-5a26-4086-822a-a77557fb4057","Type":"ContainerStarted","Data":"12872c7fa493948db89569beaf95ca51739135087c722d4b85c1fb2d84835948"}
Sep 18 18:44:34.060963 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.060937    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vdgtt" event={"ID":"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4","Type":"ContainerStarted","Data":"6a20f51232421ca096c0d522738713b85901f48b2fd42efde36038fb5420e3c5"}
Sep 18 18:44:34.064555 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.062634    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nsfwm" event={"ID":"e8386b94-b143-4f08-a95a-ce6fb840c8f3","Type":"ContainerStarted","Data":"db3b3812fc7a537cb78dccbaf9d6722559eb018aa699c132eaa174afcdf79811"}
Sep 18 18:44:34.066375 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.066355    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerStarted","Data":"45bcf245a74690021fc3246df972d0328b6e920644abedbccba0762c76b748f0"}
Sep 18 18:44:34.067702 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.067578    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqmtv" event={"ID":"42dc14ed-ebaa-4847-91f3-acd381315150","Type":"ContainerStarted","Data":"f0e2bdc9f6200e07593646ad6a60777179c48cf39bbb35dc7f113e61d5c6d488"}
Sep 18 18:44:34.068979 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.068955    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"b8cef0c55e7152a356169e9766ae9968c6e173b7ef5974b8af537b4f1195c00c"}
Sep 18 18:44:34.070212 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.070192    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nfq7b" event={"ID":"d3e57d49-e5b9-454c-925c-24f923cf8c69","Type":"ContainerStarted","Data":"6f931db82ac9b50890e61774d38d42fb5cd6209baddafb5ca0dcdb610769fc77"}
Sep 18 18:44:34.071422 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.071378    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-77jzc" event={"ID":"3d4a16e1-bc0d-4298-8762-493d4d442b73","Type":"ContainerStarted","Data":"ace41ccaa303230cfc44fc7179b983516640a84bec94da943e1a7ff8b7e397e7"}
Sep 18 18:44:34.457430 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.457401    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:44:34.547703 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.547618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:34.547858 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.547750    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:34.547858 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.547812    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:44:36.547793972 +0000 UTC m=+5.986054160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:34.648262 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.648229    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:34.648262 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.648266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:34.648488 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.648363    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:34.648488 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.648377    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:34.648488 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.648386    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:34.648488 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.648434    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:44:36.648419797 +0000 UTC m=+6.086679993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:34.648762 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.648617    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:34.648762 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:34.648672    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:36.648656878 +0000 UTC m=+6.086917074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:34.967816 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.967754    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 18:39:32 +0000 UTC" deadline="2028-05-22 06:22:35.235278413 +0000 UTC"
Sep 18 18:44:34.967816 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:34.967815    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14675h38m0.26746712s"
Sep 18 18:44:35.048662 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:35.048631    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:35.048838 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:35.048763    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:35.049187 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:35.049170    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:35.049299 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:35.049275    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:35.054420 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:35.051435    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:35.054420 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:35.051538    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:35.092248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:35.092213    2568 generic.go:356] "Generic (PLEG): container finished" podID="d97b59528ec8f7f1a82a1af8817dbcf5" containerID="c75c810390b96d4402304bf1a8129e56310d3a8c4266cd3e982a880833168ed6" exitCode=0
Sep 18 18:44:35.092391 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:35.092303    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal" event={"ID":"d97b59528ec8f7f1a82a1af8817dbcf5","Type":"ContainerDied","Data":"c75c810390b96d4402304bf1a8129e56310d3a8c4266cd3e982a880833168ed6"}
Sep 18 18:44:35.111187 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:35.110952    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal" event={"ID":"8708fb89a1341e781b6f6cd3d321ee06","Type":"ContainerStarted","Data":"d003dafb45c43f6585af3139b90dfe01c341e68217dfedc2d1c2a533ab4338e1"}
Sep 18 18:44:36.146387 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:36.146351    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal" event={"ID":"d97b59528ec8f7f1a82a1af8817dbcf5","Type":"ContainerStarted","Data":"7d24251e3d2349e06dcffe205f9f30701b7d378b2c3aadbfcc426539bf49f52d"}
Sep 18 18:44:36.160504 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:36.160449    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-94.ec2.internal" podStartSLOduration=4.160431889 podStartE2EDuration="4.160431889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:36.160036379 +0000 UTC m=+5.598296576" watchObservedRunningTime="2026-09-18 18:44:36.160431889 +0000 UTC m=+5.598692094"
Sep 18 18:44:36.161037 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:36.161001    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-94.ec2.internal" podStartSLOduration=4.160990442 podStartE2EDuration="4.160990442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:35.124631637 +0000 UTC m=+4.562891843" watchObservedRunningTime="2026-09-18 18:44:36.160990442 +0000 UTC m=+5.599250648"
Sep 18 18:44:36.566025 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:36.565367    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:36.566025 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.565562    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:36.566025 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.565627    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:44:40.565606002 +0000 UTC m=+10.003866190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:36.666401    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:36.666460    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.666643    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.666664    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.666680    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.666743    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:44:40.666723377 +0000 UTC m=+10.104983591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.667126    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:36.667228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:36.667180    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:40.667157572 +0000 UTC m=+10.105417759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:37.049764 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:37.049158    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:37.049764 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:37.049290    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:37.049764 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:37.049622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:37.049764 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:37.049728    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:37.050596 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:37.050432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:37.050596 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:37.050508    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:39.048649 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:39.048609    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:39.049116 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:39.048754    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:39.049116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:39.049082    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:39.049228 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:39.049179    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:39.049280 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:39.049223    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:39.049337 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:39.049305    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:40.597732 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:40.597689    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:40.598175 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.597866    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:40.598175 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.597926    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:44:48.59791225 +0000 UTC m=+18.036172432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:40.698998 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:40.698960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:40.699174 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:40.699018    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:40.699174 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.699140    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:40.699174 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.699156    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:40.699340 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.699186    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:40.699340 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.699201    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:40.699340 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.699218    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:44:48.699197277 +0000 UTC m=+18.137457463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:40.699340 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:40.699254    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:44:48.69923675 +0000 UTC m=+18.137496943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:41.049792 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:41.049754    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:41.049983 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:41.049869    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:41.050583 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:41.050288    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:41.050583 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:41.050388    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:41.050583 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:41.050444    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:41.050583 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:41.050552    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:43.048735 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:43.048697    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:43.049105 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:43.048697    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:43.049105 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:43.048818    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:43.049105 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:43.048873    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:43.049105 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:43.048698    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:43.049105 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:43.048990    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:45.048743 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:45.048711    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:45.049210 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:45.048714    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:45.049210 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:45.048845    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:45.049210 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:45.048709    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:45.049210 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:45.048927    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:45.049210 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:45.049019    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:47.048713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:47.048442    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:47.049148 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:47.048459    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:47.049148 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:47.048820    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:47.049148 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:47.048469    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:47.049148 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:47.048953    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:47.049148 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:47.048990    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:48.660654 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:48.660609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:48.661123 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.660828    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:48.661123 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.660908    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:45:04.660890142 +0000 UTC m=+34.099150331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:44:48.761328 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:48.761291    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:48.761328 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:48.761337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:48.761601 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.761449    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:48.761601 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.761462    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:44:48.761601 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.761543    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:04.761508581 +0000 UTC m=+34.199768767 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:44:48.761601 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.761568    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:44:48.761601 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.761581    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:48.761825 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:48.761628    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:45:04.761613076 +0000 UTC m=+34.199873275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:44:49.049625 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:49.049575    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:49.049816 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:49.049712    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:49.049816 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:49.049724    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:49.049932 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:49.049822    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:49.049932 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:49.049914    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:49.050033 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:49.050000    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:51.049532 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:51.049484    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:51.049984 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:51.049593    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:51.049984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:51.049609    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:51.049984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:51.049637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:51.049984 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:51.049708    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:51.049984 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:51.049833    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:52.179202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.178963    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np" event={"ID":"6625f6df-5a26-4086-822a-a77557fb4057","Type":"ContainerStarted","Data":"22d96d32fbe5af47109b537b9347772b168656b25c63d7c7b66f229def1aa58b"}
Sep 18 18:44:52.181333 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.181276    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vdgtt" event={"ID":"1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4","Type":"ContainerStarted","Data":"cacfb9294a40efdaf36665c85b2b98bf2867ee6ec4b1d5b8f5d5ada4d0b8a79e"}
Sep 18 18:44:52.183720 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.183314    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nsfwm" event={"ID":"e8386b94-b143-4f08-a95a-ce6fb840c8f3","Type":"ContainerStarted","Data":"84dfec110c42b303f9773799fcfec9403751e634e2c5a821b3deca8396d291a2"}
Sep 18 18:44:52.186244 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.186121    2568 generic.go:356] "Generic (PLEG): container finished" podID="71b2e4cb-ff67-4498-9222-ff7604a6d1bd" containerID="dcae285f0a40a12608943647270e4d7c3b198bdaf16a8e24d2d29c63e76615c1" exitCode=0
Sep 18 18:44:52.186244 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.186198    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerDied","Data":"dcae285f0a40a12608943647270e4d7c3b198bdaf16a8e24d2d29c63e76615c1"}
Sep 18 18:44:52.190271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.189932    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqmtv" event={"ID":"42dc14ed-ebaa-4847-91f3-acd381315150","Type":"ContainerStarted","Data":"319d3f7a8ebfc11a57593e6778f98cdbdc154dd1af7d171724374a24c9a44789"}
Sep 18 18:44:52.193811 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.193787    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"58043259ee74110f60363a9704b0028dbfb7f8b8eee8a16af7f234f9404ced31"}
Sep 18 18:44:52.193894 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.193858    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"d28ae76d924bcc5c09883f1586b60be884688ebd3ed5a38298a7c85c26089ae9"}
Sep 18 18:44:52.195431 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.195406    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nfq7b" event={"ID":"d3e57d49-e5b9-454c-925c-24f923cf8c69","Type":"ContainerStarted","Data":"abf5804269d614e16e2ed538a194af8e52a24fb0213f35dc4d1758f107a9b771"}
Sep 18 18:44:52.196906 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.196886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-77jzc" event={"ID":"3d4a16e1-bc0d-4298-8762-493d4d442b73","Type":"ContainerStarted","Data":"82f53e8a4d61243fb765f3926e9e9dca9c33cd7c508df9c834702bc279a19d67"}
Sep 18 18:44:52.210437 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.210391    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vdgtt" podStartSLOduration=3.762854508 podStartE2EDuration="21.210359991s" totalImagesPullingTime="17.447505483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:52.209778451 +0000 UTC m=+21.648038656" watchObservedRunningTime="2026-09-18 18:44:52.210359991 +0000 UTC m=+21.648620197"
Sep 18 18:44:52.238312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.238222    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nfq7b" podStartSLOduration=3.806524361 podStartE2EDuration="21.238204761s" totalImagesPullingTime="17.4316804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:52.237583048 +0000 UTC m=+21.675843252" watchObservedRunningTime="2026-09-18 18:44:52.238204761 +0000 UTC m=+21.676464965"
Sep 18 18:44:52.262283 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.262243    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pqmtv" podStartSLOduration=3.8056368210000002 podStartE2EDuration="21.262228722s" totalImagesPullingTime="17.456591901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:52.261839919 +0000 UTC m=+21.700100118" watchObservedRunningTime="2026-09-18 18:44:52.262228722 +0000 UTC m=+21.700488929"
Sep 18 18:44:52.287565 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.287419    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nsfwm" podStartSLOduration=3.824416371 podStartE2EDuration="21.287401233s" totalImagesPullingTime="17.462984862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:52.286793655 +0000 UTC m=+21.725053858" watchObservedRunningTime="2026-09-18 18:44:52.287401233 +0000 UTC m=+21.725661439"
Sep 18 18:44:52.348957 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.348910    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-77jzc" podStartSLOduration=3.854686108 podStartE2EDuration="21.348895349s" totalImagesPullingTime="17.494209241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:52.348335942 +0000 UTC m=+21.786596146" watchObservedRunningTime="2026-09-18 18:44:52.348895349 +0000 UTC m=+21.787155552"
Sep 18 18:44:52.676349 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.676327    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 18:44:52.973980 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.973885    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T18:44:52.676344804Z","UUID":"a830a665-89b8-4f81-adb9-94d91179595b","Handler":null,"Name":"","Endpoint":""}
Sep 18 18:44:52.975397 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.975376    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 18:44:52.975621 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:52.975407    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 18:44:53.048412 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.048379    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:53.048594 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.048379    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:53.048594 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:53.048487    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:53.048594 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.048389    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:53.048719 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:53.048607    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:53.048719 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:53.048681    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:53.202269 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.202184    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-96x66" event={"ID":"44db16cc-8506-4ed0-b9af-78be01affad6","Type":"ContainerStarted","Data":"86e71dce94157fca839dc5eee83880fb581b03bba563fd233520f346615e3d32"}
Sep 18 18:44:53.204726 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.204692    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np" event={"ID":"6625f6df-5a26-4086-822a-a77557fb4057","Type":"ContainerStarted","Data":"551d253b05b61012d836446421820a67aee18ddbc9c135673dc944af5847c951"}
Sep 18 18:44:53.209445 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.209416    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"49313eac974dd56af97c5ac69bbcdff0e1cd431ed2a095ff01db53ee2edcf6cd"}
Sep 18 18:44:53.209581 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.209561    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"7ebf347f1a4ade0d0412248716e0b8ef0a14fbb34e4c538dcbe1625d63330591"}
Sep 18 18:44:53.209689 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.209616    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"71730715d899fe45f2a9ce27bc56cc6f7b866831544d8d1c46433cd5b297604f"}
Sep 18 18:44:53.209689 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.209644    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"923acdeed2973add7fb1c7270cf99f3dac1bb070ed5155b77f86891d766eb8f1"}
Sep 18 18:44:53.219297 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:53.219249    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-96x66" podStartSLOduration=4.765936689 podStartE2EDuration="22.219095455s" totalImagesPullingTime="17.453158766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:53.2187992 +0000 UTC m=+22.657059416" watchObservedRunningTime="2026-09-18 18:44:53.219095455 +0000 UTC m=+22.657355659"
Sep 18 18:44:54.215991 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:54.215940    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np" event={"ID":"6625f6df-5a26-4086-822a-a77557fb4057","Type":"ContainerStarted","Data":"5bbc5b0bc4c47eec9e191e714da44a1cf42b2edb0723e8b82f740bed96664b23"}
Sep 18 18:44:54.240469 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:54.240411    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m76np" podStartSLOduration=4.197821216 podStartE2EDuration="23.240396442s" totalImagesPullingTime="19.042575226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:54.239748105 +0000 UTC m=+23.678008308" watchObservedRunningTime="2026-09-18 18:44:54.240396442 +0000 UTC m=+23.678656645"
Sep 18 18:44:54.366615 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:54.366567    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:54.367381 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:54.367357    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:55.048420 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:55.048385    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:55.048705 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:55.048388    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:55.048705 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:55.048387    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:55.048705 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:55.048532    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:55.048705 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:55.048599    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:55.048705 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:55.048686    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:55.222100 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:55.222050    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"a6d98bc0fc86c237249c9beaaace4122d5d7e20580bc0b65cdb9ad9b0c4db260"}
Sep 18 18:44:55.222667 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:55.222649    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:55.222946 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:55.222926    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nfq7b"
Sep 18 18:44:57.048083 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.048008    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:57.048492 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.048008    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:57.048492 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:57.048154    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:57.048492 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.048012    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:57.048492 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:57.048192    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:57.048492 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:57.048271    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:57.228699 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.228490    2568 generic.go:356] "Generic (PLEG): container finished" podID="71b2e4cb-ff67-4498-9222-ff7604a6d1bd" containerID="06b831b8b9053162b0f475b918b278fd770e0803a71cff5f335870fd1dfc8510" exitCode=0
Sep 18 18:44:57.228857 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.228550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerDied","Data":"06b831b8b9053162b0f475b918b278fd770e0803a71cff5f335870fd1dfc8510"}
Sep 18 18:44:57.232414 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.232387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" event={"ID":"a3959247-ef44-4f78-9fd2-4f64325cc885","Type":"ContainerStarted","Data":"9500fe88ea78f8205814718982b1282cd3ce4cbb50496bac5226dde44cee64e1"}
Sep 18 18:44:57.232869 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.232682    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:57.232869 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.232754    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:57.232869 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.232801    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:57.248499 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.248472    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:57.248716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.248634    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:44:57.270137 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:57.270092    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq" podStartSLOduration=8.274254585 podStartE2EDuration="26.270077644s" totalImagesPullingTime="17.995823059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:44:57.269121025 +0000 UTC m=+26.707381239" watchObservedRunningTime="2026-09-18 18:44:57.270077644 +0000 UTC m=+26.708337848"
Sep 18 18:44:58.237253 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.237221    2568 generic.go:356] "Generic (PLEG): container finished" podID="71b2e4cb-ff67-4498-9222-ff7604a6d1bd" containerID="2342b2cd7e50b630a41d63ff50f31cf4e11a74aebd74224c9c7f939fecb7e200" exitCode=0
Sep 18 18:44:58.237861 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.237309    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerDied","Data":"2342b2cd7e50b630a41d63ff50f31cf4e11a74aebd74224c9c7f939fecb7e200"}
Sep 18 18:44:58.678211 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.677822    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hq4lf"]
Sep 18 18:44:58.678211 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.677926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k77tr"]
Sep 18 18:44:58.678211 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.678058    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:44:58.678534 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.678493    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:44:58.678612 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:58.678469    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:44:58.679046 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:58.679017    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:44:58.686918 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.686888    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k5248"]
Sep 18 18:44:58.687031 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:58.687000    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:44:58.687100 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:44:58.687083    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:44:59.242418 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:59.242236    2568 generic.go:356] "Generic (PLEG): container finished" podID="71b2e4cb-ff67-4498-9222-ff7604a6d1bd" containerID="cd40ae79931d85a48ae50f05ac5d88fa11646c9eba541eb9f376fcf5ffe613c7" exitCode=0
Sep 18 18:44:59.243000 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:44:59.242323    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerDied","Data":"cd40ae79931d85a48ae50f05ac5d88fa11646c9eba541eb9f376fcf5ffe613c7"}
Sep 18 18:45:00.048053 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:00.048019    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:00.048222 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:00.048142    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:45:01.049803 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:01.049770    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:01.050243 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:01.049884    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:45:01.050243 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:01.049938    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:01.050243 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:01.050066    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:45:02.048232 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:02.048200    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:02.048404 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:02.048299    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:45:03.049033 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:03.048468    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:03.049033 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:03.048480    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:03.049033 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:03.048637    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:45:03.049033 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:03.048768    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:45:04.048559 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:04.048506    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:04.048720 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.048653    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k5248" podUID="f3e567a1-85ce-4000-908f-eb34b464ae1f"
Sep 18 18:45:04.256538 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:04.256455    2568 generic.go:356] "Generic (PLEG): container finished" podID="71b2e4cb-ff67-4498-9222-ff7604a6d1bd" containerID="a2e846ed20d085854cd9a7e7c0ee01040605625c69332cc467980455e81327ba" exitCode=0
Sep 18 18:45:04.256538 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:04.256507    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerDied","Data":"a2e846ed20d085854cd9a7e7c0ee01040605625c69332cc467980455e81327ba"}
Sep 18 18:45:04.669938 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:04.669854    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:04.670078 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.669970    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:45:04.670078 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.670016    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret podName:51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd nodeName:}" failed. No retries permitted until 2026-09-18 18:45:36.670003424 +0000 UTC m=+66.108263605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret") pod "global-pull-secret-syncer-hq4lf" (UID: "51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:45:04.771142 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:04.771105    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:04.771142 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:04.771142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:04.771330 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.771247    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:45:04.771330 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.771265    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:45:04.771330 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.771273    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p4dqh for pod openshift-network-diagnostics/network-check-target-k5248: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:45:04.771330 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.771312    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh podName:f3e567a1-85ce-4000-908f-eb34b464ae1f nodeName:}" failed. No retries permitted until 2026-09-18 18:45:36.771300705 +0000 UTC m=+66.209560887 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p4dqh" (UniqueName: "kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh") pod "network-check-target-k5248" (UID: "f3e567a1-85ce-4000-908f-eb34b464ae1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:45:04.771330 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.771246    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:45:04.771476 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:04.771381    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs podName:fe46cc04-7e73-4bcf-8a60-b684fb221935 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:36.771366896 +0000 UTC m=+66.209627077 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs") pod "network-metrics-daemon-k77tr" (UID: "fe46cc04-7e73-4bcf-8a60-b684fb221935") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:45:05.049119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.049097    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:05.049218 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:05.049201    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hq4lf" podUID="51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd"
Sep 18 18:45:05.049274 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.049254    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:05.049375 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:05.049354    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k77tr" podUID="fe46cc04-7e73-4bcf-8a60-b684fb221935"
Sep 18 18:45:05.262008 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.261976    2568 generic.go:356] "Generic (PLEG): container finished" podID="71b2e4cb-ff67-4498-9222-ff7604a6d1bd" containerID="ba62067cc1d701317778b01ed1f2b2bbc81d9ec8040aa9bda7742a04333fae48" exitCode=0
Sep 18 18:45:05.262426 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.262044    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerDied","Data":"ba62067cc1d701317778b01ed1f2b2bbc81d9ec8040aa9bda7742a04333fae48"}
Sep 18 18:45:05.275168 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.275115    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-94.ec2.internal" event="NodeReady"
Sep 18 18:45:05.275240 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.275230    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 18:45:05.319000 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.318970    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qgl2s"]
Sep 18 18:45:05.349799 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.349777    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kfx72"]
Sep 18 18:45:05.349924 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.349909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.352682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.352659    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 18:45:05.352805 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.352694    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qf4nw\""
Sep 18 18:45:05.352805 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.352696    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 18:45:05.377026 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.376999    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qgl2s"]
Sep 18 18:45:05.377026 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.377033    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfx72"]
Sep 18 18:45:05.377188 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.377100    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.379884 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.379864    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sl7vh\""
Sep 18 18:45:05.379884 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.379866    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 18:45:05.380586 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.380573    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 18:45:05.381127 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.381113    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 18:45:05.476231 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.476191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.476231 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.476235    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dmmh\" (UniqueName: \"kubernetes.io/projected/33113daa-1aea-42d9-82bd-ecaa8d22cc9a-kube-api-access-2dmmh\") pod \"ingress-canary-kfx72\" (UID: \"33113daa-1aea-42d9-82bd-ecaa8d22cc9a\") " pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.476602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.476269    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48b6c68-2dc3-4f03-b60a-1b181429c33e-tmp-dir\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.476602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.476334    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33113daa-1aea-42d9-82bd-ecaa8d22cc9a-cert\") pod \"ingress-canary-kfx72\" (UID: \"33113daa-1aea-42d9-82bd-ecaa8d22cc9a\") " pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.476602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.476372    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a48b6c68-2dc3-4f03-b60a-1b181429c33e-config-volume\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.476602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.476420    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvcnm\" (UniqueName: \"kubernetes.io/projected/a48b6c68-2dc3-4f03-b60a-1b181429c33e-kube-api-access-rvcnm\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.577156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.577156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dmmh\" (UniqueName: \"kubernetes.io/projected/33113daa-1aea-42d9-82bd-ecaa8d22cc9a-kube-api-access-2dmmh\") pod \"ingress-canary-kfx72\" (UID: \"33113daa-1aea-42d9-82bd-ecaa8d22cc9a\") " pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.577156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577148    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48b6c68-2dc3-4f03-b60a-1b181429c33e-tmp-dir\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.577384 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577186    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33113daa-1aea-42d9-82bd-ecaa8d22cc9a-cert\") pod \"ingress-canary-kfx72\" (UID: \"33113daa-1aea-42d9-82bd-ecaa8d22cc9a\") " pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.577384 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577208    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a48b6c68-2dc3-4f03-b60a-1b181429c33e-config-volume\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.577384 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:05.577234    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:05.577384 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577251    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvcnm\" (UniqueName: \"kubernetes.io/projected/a48b6c68-2dc3-4f03-b60a-1b181429c33e-kube-api-access-rvcnm\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.577384 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:05.577309    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls podName:a48b6c68-2dc3-4f03-b60a-1b181429c33e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:06.077285268 +0000 UTC m=+35.515545453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls") pod "dns-default-qgl2s" (UID: "a48b6c68-2dc3-4f03-b60a-1b181429c33e") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:05.577684 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48b6c68-2dc3-4f03-b60a-1b181429c33e-tmp-dir\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.577791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.577771    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a48b6c68-2dc3-4f03-b60a-1b181429c33e-config-volume\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.581424 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.581403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33113daa-1aea-42d9-82bd-ecaa8d22cc9a-cert\") pod \"ingress-canary-kfx72\" (UID: \"33113daa-1aea-42d9-82bd-ecaa8d22cc9a\") " pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.592010 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.591989    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dmmh\" (UniqueName: \"kubernetes.io/projected/33113daa-1aea-42d9-82bd-ecaa8d22cc9a-kube-api-access-2dmmh\") pod \"ingress-canary-kfx72\" (UID: \"33113daa-1aea-42d9-82bd-ecaa8d22cc9a\") " pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:05.592010 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.592005    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvcnm\" (UniqueName: \"kubernetes.io/projected/a48b6c68-2dc3-4f03-b60a-1b181429c33e-kube-api-access-rvcnm\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:05.685384 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:05.685348    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfx72"
Sep 18 18:45:06.021968 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.021929    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfx72"]
Sep 18 18:45:06.026267 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:06.026243    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33113daa_1aea_42d9_82bd_ecaa8d22cc9a.slice/crio-dcb4da9bc833d34c639e876b5b3bbab8c6ce92eec19e2bba953a930fba386a44 WatchSource:0}: Error finding container dcb4da9bc833d34c639e876b5b3bbab8c6ce92eec19e2bba953a930fba386a44: Status 404 returned error can't find the container with id dcb4da9bc833d34c639e876b5b3bbab8c6ce92eec19e2bba953a930fba386a44
Sep 18 18:45:06.049005 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.048983    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:06.053074 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.053054    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:45:06.053169 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.053054    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:45:06.053169 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.053116    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vk59r\""
Sep 18 18:45:06.081798 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.081772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:06.081930 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:06.081907    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:06.081976 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:06.081964    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls podName:a48b6c68-2dc3-4f03-b60a-1b181429c33e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:07.081949766 +0000 UTC m=+36.520209948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls") pod "dns-default-qgl2s" (UID: "a48b6c68-2dc3-4f03-b60a-1b181429c33e") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:06.267065 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.267030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" event={"ID":"71b2e4cb-ff67-4498-9222-ff7604a6d1bd","Type":"ContainerStarted","Data":"a4dff4b85f989a4e35a3abdc1e072f9af3c01f22d020d81ed7c87b32796d556c"}
Sep 18 18:45:06.268198 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.268175    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfx72" event={"ID":"33113daa-1aea-42d9-82bd-ecaa8d22cc9a","Type":"ContainerStarted","Data":"dcb4da9bc833d34c639e876b5b3bbab8c6ce92eec19e2bba953a930fba386a44"}
Sep 18 18:45:06.295367 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:06.295279    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k5z4l" podStartSLOduration=7.575522959 podStartE2EDuration="35.295264216s" totalImagesPullingTime="27.719741257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:06.293877741 +0000 UTC m=+35.732137943" watchObservedRunningTime="2026-09-18 18:45:06.295264216 +0000 UTC m=+35.733524418"
Sep 18 18:45:07.048429 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:07.048392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:07.048700 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:07.048541    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:07.050989 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:07.050963    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:45:07.051115 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:07.051027    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xdsv4\""
Sep 18 18:45:07.051115 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:07.050966    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:45:07.088804 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:07.088772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:07.088977 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:07.088952    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:07.089097 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:07.089027    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls podName:a48b6c68-2dc3-4f03-b60a-1b181429c33e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:09.089008041 +0000 UTC m=+38.527268228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls") pod "dns-default-qgl2s" (UID: "a48b6c68-2dc3-4f03-b60a-1b181429c33e") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:09.103759 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.103567    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:09.104129 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:09.103724    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:09.104129 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:09.103869    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls podName:a48b6c68-2dc3-4f03-b60a-1b181429c33e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.10385442 +0000 UTC m=+42.542114601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls") pod "dns-default-qgl2s" (UID: "a48b6c68-2dc3-4f03-b60a-1b181429c33e") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:09.277661 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.277626    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfx72" event={"ID":"33113daa-1aea-42d9-82bd-ecaa8d22cc9a","Type":"ContainerStarted","Data":"6c2a11d627e67d7a0725aeb5a061fdd8a1282a6bd2d4e9f1e03a81811e87a06b"}
Sep 18 18:45:09.716974 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.716920    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kfx72" podStartSLOduration=2.036836252 podStartE2EDuration="4.716900517s" totalImagesPullingTime="2.680064265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:09.304255319 +0000 UTC m=+38.742515524" watchObservedRunningTime="2026-09-18 18:45:09.716900517 +0000 UTC m=+39.155160718"
Sep 18 18:45:09.717798 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.717776    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"]
Sep 18 18:45:09.732716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.732678    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"
Sep 18 18:45:09.732915 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.732893    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"]
Sep 18 18:45:09.736064 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.736044    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xtsdd\""
Sep 18 18:45:09.808606 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.808569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsf2x\" (UniqueName: \"kubernetes.io/projected/db81ce56-291c-4a0e-8c02-45d2eba37309-kube-api-access-bsf2x\") pod \"network-check-source-7cbc45fddc-nqz29\" (UID: \"db81ce56-291c-4a0e-8c02-45d2eba37309\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"
Sep 18 18:45:09.909266 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.909231    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsf2x\" (UniqueName: \"kubernetes.io/projected/db81ce56-291c-4a0e-8c02-45d2eba37309-kube-api-access-bsf2x\") pod \"network-check-source-7cbc45fddc-nqz29\" (UID: \"db81ce56-291c-4a0e-8c02-45d2eba37309\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"
Sep 18 18:45:09.931065 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.931029    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsf2x\" (UniqueName: \"kubernetes.io/projected/db81ce56-291c-4a0e-8c02-45d2eba37309-kube-api-access-bsf2x\") pod \"network-check-source-7cbc45fddc-nqz29\" (UID: \"db81ce56-291c-4a0e-8c02-45d2eba37309\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"
Sep 18 18:45:09.957601 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.957569    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"]
Sep 18 18:45:09.984570 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.984478    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cbd59bc79-h9mc7"]
Sep 18 18:45:09.984720 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.984587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:09.987258 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.987238    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 18 18:45:09.987561 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.987543    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 18 18:45:09.987873 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.987856    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 18 18:45:09.987954 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.987906    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:45:09.988100 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:09.988082    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-s769m\""
Sep 18 18:45:10.008764 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.008728    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8587f4c686-59cps"]
Sep 18 18:45:10.008892 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.008843    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.011267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.011247    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m6jx4\""
Sep 18 18:45:10.011465 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.011452    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 18:45:10.011991 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.011977    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 18:45:10.013637 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.013622    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 18:45:10.018405 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.018385    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 18:45:10.028912 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.028894    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-dncfj"]
Sep 18 18:45:10.029019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.029010    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.034218 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.034203    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 18 18:45:10.034922 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.034904    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:45:10.035025 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.035007    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 18 18:45:10.035123 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.035107    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2zbnj\""
Sep 18 18:45:10.035180 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.035116    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:45:10.035568 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.035555    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 18 18:45:10.035625 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.035599    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 18 18:45:10.041553 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.041537    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"
Sep 18 18:45:10.052936 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.052916    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"]
Sep 18 18:45:10.053028 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.052949    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbd59bc79-h9mc7"]
Sep 18 18:45:10.053028 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.052970    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8587f4c686-59cps"]
Sep 18 18:45:10.053028 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.052988    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-dncfj"]
Sep 18 18:45:10.053028 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.053019    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-b9nxp"]
Sep 18 18:45:10.053206 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.053028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.056551 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.056535    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 18:45:10.056650 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.056598    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 18:45:10.056650 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.056633    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:45:10.056754 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.056735    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 18:45:10.057896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.057877    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hhrv5\""
Sep 18 18:45:10.075291 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.075265    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-b9nxp"]
Sep 18 18:45:10.075471 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.075355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.081850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.081078    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 18:45:10.081850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.081235    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 18 18:45:10.081850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.081376    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 18:45:10.081850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.081566    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 18:45:10.081850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.081767    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qxzg2\""
Sep 18 18:45:10.082168 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.081895    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 18 18:45:10.088969 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.088950    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 18 18:45:10.110528 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110442    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-trusted-ca\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.110850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110544    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpfnj\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-kube-api-access-fpfnj\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.110850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-stats-auth\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.110850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110661    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa841efa-e805-4e1a-85e9-db1c65514614-config\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.110850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110774    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5456\" (UniqueName: \"kubernetes.io/projected/fa841efa-e805-4e1a-85e9-db1c65514614-kube-api-access-z5456\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.111002 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110852    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.111002 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110889    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-bound-sa-token\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.111002 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.110951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-installation-pull-secrets\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.111116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111005    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa841efa-e805-4e1a-85e9-db1c65514614-serving-cert\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.111116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111033    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.111116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111086    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhjcs\" (UniqueName: \"kubernetes.io/projected/85c147de-2421-41c8-b625-33d6a9cb1e58-kube-api-access-jhjcs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.111244 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111140    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-image-registry-private-configuration\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.111244 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-certificates\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.111244 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111214    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-default-certificate\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.111362 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111272    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.111362 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.111302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cb38fe2-b1e1-47a5-ae73-504df175336e-ca-trust-extracted\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.212205 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-stats-auth\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.212205 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212210    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzgt6\" (UniqueName: \"kubernetes.io/projected/3125a1e7-a100-4427-b812-ca56cf0ef2be-kube-api-access-gzgt6\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212228    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3125a1e7-a100-4427-b812-ca56cf0ef2be-trusted-ca\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212258    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b078765-14df-41e5-af2c-08bd9f34612e-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212281    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212330    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3125a1e7-a100-4427-b812-ca56cf0ef2be-config\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhjcs\" (UniqueName: \"kubernetes.io/projected/85c147de-2421-41c8-b625-33d6a9cb1e58-kube-api-access-jhjcs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212398    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3125a1e7-a100-4427-b812-ca56cf0ef2be-serving-cert\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212415    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa841efa-e805-4e1a-85e9-db1c65514614-serving-cert\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212431    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.212435    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:45:10.212455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212450    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-image-registry-private-configuration\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.212506    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:10.712483185 +0000 UTC m=+40.150743367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : secret "router-metrics-certs-default" not found
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212551    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-655vp\" (UniqueName: \"kubernetes.io/projected/0b078765-14df-41e5-af2c-08bd9f34612e-kube-api-access-655vp\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.212560    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.212591    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbd59bc79-h9mc7: secret "image-registry-tls" not found
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.212591    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:10.712577158 +0000 UTC m=+40.150837362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : configmap references non-existent config key: service-ca.crt
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212639    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cb38fe2-b1e1-47a5-ae73-504df175336e-ca-trust-extracted\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b078765-14df-41e5-af2c-08bd9f34612e-snapshots\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.212707    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls podName:4cb38fe2-b1e1-47a5-ae73-504df175336e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:10.712688949 +0000 UTC m=+40.150949139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls") pod "image-registry-7cbd59bc79-h9mc7" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e") : secret "image-registry-tls" not found
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpfnj\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-kube-api-access-fpfnj\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212792    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b078765-14df-41e5-af2c-08bd9f34612e-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212820    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5456\" (UniqueName: \"kubernetes.io/projected/fa841efa-e805-4e1a-85e9-db1c65514614-kube-api-access-z5456\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212843    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b078765-14df-41e5-af2c-08bd9f34612e-tmp\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212874    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa841efa-e805-4e1a-85e9-db1c65514614-config\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.212988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-bound-sa-token\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212934    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-installation-pull-secrets\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.212974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-certificates\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.213015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-default-certificate\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.213053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b078765-14df-41e5-af2c-08bd9f34612e-serving-cert\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.213072    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cb38fe2-b1e1-47a5-ae73-504df175336e-ca-trust-extracted\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.213107    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-trusted-ca\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.213742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.213590    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-certificates\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.214065 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.213797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa841efa-e805-4e1a-85e9-db1c65514614-config\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.214065 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.214045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-trusted-ca\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.214840 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.214814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa841efa-e805-4e1a-85e9-db1c65514614-serving-cert\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.215375 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.215350    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-stats-auth\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.215835 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.215816    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-installation-pull-secrets\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.215835 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.215828    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-default-certificate\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.216017 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.215990    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-image-registry-private-configuration\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.232601 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.232576    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-bound-sa-token\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.234066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.234039    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpfnj\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-kube-api-access-fpfnj\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.234341 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.234324    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhjcs\" (UniqueName: \"kubernetes.io/projected/85c147de-2421-41c8-b625-33d6a9cb1e58-kube-api-access-jhjcs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.235409 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.235371    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5456\" (UniqueName: \"kubernetes.io/projected/fa841efa-e805-4e1a-85e9-db1c65514614-kube-api-access-z5456\") pod \"service-ca-operator-57cdc4b768-ppl49\" (UID: \"fa841efa-e805-4e1a-85e9-db1c65514614\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.293129 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.293100    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"
Sep 18 18:45:10.313980 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.313953    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b078765-14df-41e5-af2c-08bd9f34612e-snapshots\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.314131 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.313988    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b078765-14df-41e5-af2c-08bd9f34612e-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.314131 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314005    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b078765-14df-41e5-af2c-08bd9f34612e-tmp\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.314131 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b078765-14df-41e5-af2c-08bd9f34612e-serving-cert\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.314131 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzgt6\" (UniqueName: \"kubernetes.io/projected/3125a1e7-a100-4427-b812-ca56cf0ef2be-kube-api-access-gzgt6\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.314131 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314102    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3125a1e7-a100-4427-b812-ca56cf0ef2be-trusted-ca\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.314372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b078765-14df-41e5-af2c-08bd9f34612e-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.314372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314190    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3125a1e7-a100-4427-b812-ca56cf0ef2be-config\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.314372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314219    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3125a1e7-a100-4427-b812-ca56cf0ef2be-serving-cert\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.314372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-655vp\" (UniqueName: \"kubernetes.io/projected/0b078765-14df-41e5-af2c-08bd9f34612e-kube-api-access-655vp\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.315308 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314805    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b078765-14df-41e5-af2c-08bd9f34612e-snapshots\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.315308 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.314879    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b078765-14df-41e5-af2c-08bd9f34612e-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.315308 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.315269    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b078765-14df-41e5-af2c-08bd9f34612e-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.315308 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.315274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3125a1e7-a100-4427-b812-ca56cf0ef2be-trusted-ca\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.315308 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.315309    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3125a1e7-a100-4427-b812-ca56cf0ef2be-config\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.315691 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.315557    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b078765-14df-41e5-af2c-08bd9f34612e-tmp\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.317468 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.317452    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b078765-14df-41e5-af2c-08bd9f34612e-serving-cert\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.317546 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.317456    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3125a1e7-a100-4427-b812-ca56cf0ef2be-serving-cert\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.332057 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.332035    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-655vp\" (UniqueName: \"kubernetes.io/projected/0b078765-14df-41e5-af2c-08bd9f34612e-kube-api-access-655vp\") pod \"insights-operator-6f9c644dd9-b9nxp\" (UID: \"0b078765-14df-41e5-af2c-08bd9f34612e\") " pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.332146 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.332084    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzgt6\" (UniqueName: \"kubernetes.io/projected/3125a1e7-a100-4427-b812-ca56cf0ef2be-kube-api-access-gzgt6\") pod \"console-operator-fd4bb46c-dncfj\" (UID: \"3125a1e7-a100-4427-b812-ca56cf0ef2be\") " pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.339685 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.339663    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29"]
Sep 18 18:45:10.342703 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:10.342683    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb81ce56_291c_4a0e_8c02_45d2eba37309.slice/crio-04500d8a3f1831e757bc65830c45baaa3bea8ac0bfd968806dab0f936b50160e WatchSource:0}: Error finding container 04500d8a3f1831e757bc65830c45baaa3bea8ac0bfd968806dab0f936b50160e: Status 404 returned error can't find the container with id 04500d8a3f1831e757bc65830c45baaa3bea8ac0bfd968806dab0f936b50160e
Sep 18 18:45:10.360823 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.360787    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:10.388549 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.388511    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp"
Sep 18 18:45:10.591700 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.591617    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49"]
Sep 18 18:45:10.595209 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:10.595169    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa841efa_e805_4e1a_85e9_db1c65514614.slice/crio-e54263dfdadf524f3de7016b6892eeb256a72ca66384b1b79d7495c4ed0c00d4 WatchSource:0}: Error finding container e54263dfdadf524f3de7016b6892eeb256a72ca66384b1b79d7495c4ed0c00d4: Status 404 returned error can't find the container with id e54263dfdadf524f3de7016b6892eeb256a72ca66384b1b79d7495c4ed0c00d4
Sep 18 18:45:10.666985 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.666949    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-dncfj"]
Sep 18 18:45:10.670036 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:10.670010    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3125a1e7_a100_4427_b812_ca56cf0ef2be.slice/crio-eae5839ea77d544d66e9ce0e28a0c83d61136a1dd816f13653dbe7493499a3f2 WatchSource:0}: Error finding container eae5839ea77d544d66e9ce0e28a0c83d61136a1dd816f13653dbe7493499a3f2: Status 404 returned error can't find the container with id eae5839ea77d544d66e9ce0e28a0c83d61136a1dd816f13653dbe7493499a3f2
Sep 18 18:45:10.696896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.696865    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-b9nxp"]
Sep 18 18:45:10.700610 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:10.700578    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b078765_14df_41e5_af2c_08bd9f34612e.slice/crio-5341f9d1520f13cdb8fb4f783320acbca88ee8dec2c5532c5cd310643e5dd7e1 WatchSource:0}: Error finding container 5341f9d1520f13cdb8fb4f783320acbca88ee8dec2c5532c5cd310643e5dd7e1: Status 404 returned error can't find the container with id 5341f9d1520f13cdb8fb4f783320acbca88ee8dec2c5532c5cd310643e5dd7e1
Sep 18 18:45:10.718062 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.718042    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.718164 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.718076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:10.718164 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:10.718096    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:10.718239 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.718187    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:45:10.718239 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.718191    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:10.718239 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.718232    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbd59bc79-h9mc7: secret "image-registry-tls" not found
Sep 18 18:45:10.718334 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.718221    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:11.718208259 +0000 UTC m=+41.156468443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : configmap references non-existent config key: service-ca.crt
Sep 18 18:45:10.718334 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.718268    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:11.71825655 +0000 UTC m=+41.156516731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : secret "router-metrics-certs-default" not found
Sep 18 18:45:10.718334 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:10.718284    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls podName:4cb38fe2-b1e1-47a5-ae73-504df175336e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:11.71827889 +0000 UTC m=+41.156539071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls") pod "image-registry-7cbd59bc79-h9mc7" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e") : secret "image-registry-tls" not found
Sep 18 18:45:11.283766 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.283725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49" event={"ID":"fa841efa-e805-4e1a-85e9-db1c65514614","Type":"ContainerStarted","Data":"e54263dfdadf524f3de7016b6892eeb256a72ca66384b1b79d7495c4ed0c00d4"}
Sep 18 18:45:11.285042 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.285010    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" event={"ID":"3125a1e7-a100-4427-b812-ca56cf0ef2be","Type":"ContainerStarted","Data":"eae5839ea77d544d66e9ce0e28a0c83d61136a1dd816f13653dbe7493499a3f2"}
Sep 18 18:45:11.286240 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.286214    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp" event={"ID":"0b078765-14df-41e5-af2c-08bd9f34612e","Type":"ContainerStarted","Data":"5341f9d1520f13cdb8fb4f783320acbca88ee8dec2c5532c5cd310643e5dd7e1"}
Sep 18 18:45:11.287387 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.287365    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29" event={"ID":"db81ce56-291c-4a0e-8c02-45d2eba37309","Type":"ContainerStarted","Data":"04500d8a3f1831e757bc65830c45baaa3bea8ac0bfd968806dab0f936b50160e"}
Sep 18 18:45:11.726263 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.726230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:11.726440 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.726323    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:11.726440 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:11.726373    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:11.726440 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:11.726399    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:45:11.726627 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:11.726459    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.72644331 +0000 UTC m=+43.164703497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : secret "router-metrics-certs-default" not found
Sep 18 18:45:11.726627 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:11.726528    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:11.726627 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:11.726547    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbd59bc79-h9mc7: secret "image-registry-tls" not found
Sep 18 18:45:11.726627 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:11.726589    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.726573711 +0000 UTC m=+43.164833896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : configmap references non-existent config key: service-ca.crt
Sep 18 18:45:11.726627 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:11.726605    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls podName:4cb38fe2-b1e1-47a5-ae73-504df175336e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:13.726598444 +0000 UTC m=+43.164858625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls") pod "image-registry-7cbd59bc79-h9mc7" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e") : secret "image-registry-tls" not found
Sep 18 18:45:13.138379 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:13.138338    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:13.138826 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.138551    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:13.138826 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.138619    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls podName:a48b6c68-2dc3-4f03-b60a-1b181429c33e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:21.138604328 +0000 UTC m=+50.576864509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls") pod "dns-default-qgl2s" (UID: "a48b6c68-2dc3-4f03-b60a-1b181429c33e") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:13.743060 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:13.743021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:13.743060 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:13.743064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:13.743252 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:13.743081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:13.743252 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.743180    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:45:13.743252 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.743199    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:17.743185882 +0000 UTC m=+47.181446063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : configmap references non-existent config key: service-ca.crt
Sep 18 18:45:13.743252 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.743233    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:17.74321992 +0000 UTC m=+47.181480102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : secret "router-metrics-certs-default" not found
Sep 18 18:45:13.743252 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.743183    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:13.743252 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.743248    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbd59bc79-h9mc7: secret "image-registry-tls" not found
Sep 18 18:45:13.743490 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:13.743274    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls podName:4cb38fe2-b1e1-47a5-ae73-504df175336e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:17.743268509 +0000 UTC m=+47.181528690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls") pod "image-registry-7cbd59bc79-h9mc7" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e") : secret "image-registry-tls" not found
Sep 18 18:45:16.304825 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.304784    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49" event={"ID":"fa841efa-e805-4e1a-85e9-db1c65514614","Type":"ContainerStarted","Data":"0332cf1d97f515fa261a853054b9fb21b1e53bd9e4c654e9c6b9f68d53c05438"}
Sep 18 18:45:16.306466 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.306448    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/0.log"
Sep 18 18:45:16.306595 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.306481    2568 generic.go:356] "Generic (PLEG): container finished" podID="3125a1e7-a100-4427-b812-ca56cf0ef2be" containerID="8811310155cdbb95c47adca21de3416fd6444e96393693782ed384a63ec5a4b4" exitCode=255
Sep 18 18:45:16.306595 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.306510    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" event={"ID":"3125a1e7-a100-4427-b812-ca56cf0ef2be","Type":"ContainerDied","Data":"8811310155cdbb95c47adca21de3416fd6444e96393693782ed384a63ec5a4b4"}
Sep 18 18:45:16.306781 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.306764    2568 scope.go:117] "RemoveContainer" containerID="8811310155cdbb95c47adca21de3416fd6444e96393693782ed384a63ec5a4b4"
Sep 18 18:45:16.308835 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.308815    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp" event={"ID":"0b078765-14df-41e5-af2c-08bd9f34612e","Type":"ContainerStarted","Data":"d5daaae4b04244d479fcf050391cef195aaa6ca92181cd31b9ae77905ce1f603"}
Sep 18 18:45:16.311121 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.310872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29" event={"ID":"db81ce56-291c-4a0e-8c02-45d2eba37309","Type":"ContainerStarted","Data":"7b32f15d85b060f0e0e0267154164f2fad2e73a1d20dcf1c94ecdc8a6563a8e9"}
Sep 18 18:45:16.324956 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.324910    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49" podStartSLOduration=2.013333233 podStartE2EDuration="7.32489601s" totalImagesPullingTime="5.311562777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:16.324331257 +0000 UTC m=+45.762591461" watchObservedRunningTime="2026-09-18 18:45:16.32489601 +0000 UTC m=+45.763156206"
Sep 18 18:45:16.341008 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.340961    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-nqz29" podStartSLOduration=1.767276164 podStartE2EDuration="7.34094294s" totalImagesPullingTime="5.573666776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:16.339649011 +0000 UTC m=+45.777909214" watchObservedRunningTime="2026-09-18 18:45:16.34094294 +0000 UTC m=+45.779203144"
Sep 18 18:45:16.370171 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:16.370121    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp" podStartSLOduration=1.163713461 podStartE2EDuration="6.370104117s" totalImagesPullingTime="5.206390656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:16.369509277 +0000 UTC m=+45.807769480" watchObservedRunningTime="2026-09-18 18:45:16.370104117 +0000 UTC m=+45.808364317"
Sep 18 18:45:17.317982 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.317954    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:45:17.318457 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.318440    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/0.log"
Sep 18 18:45:17.318585 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.318557    2568 generic.go:356] "Generic (PLEG): container finished" podID="3125a1e7-a100-4427-b812-ca56cf0ef2be" containerID="2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e" exitCode=255
Sep 18 18:45:17.318668 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.318558    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" event={"ID":"3125a1e7-a100-4427-b812-ca56cf0ef2be","Type":"ContainerDied","Data":"2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e"}
Sep 18 18:45:17.318668 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.318661    2568 scope.go:117] "RemoveContainer" containerID="8811310155cdbb95c47adca21de3416fd6444e96393693782ed384a63ec5a4b4"
Sep 18 18:45:17.319242 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.318854    2568 scope.go:117] "RemoveContainer" containerID="2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e"
Sep 18 18:45:17.319242 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.319013    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-dncfj_openshift-console-operator(3125a1e7-a100-4427-b812-ca56cf0ef2be)\"" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" podUID="3125a1e7-a100-4427-b812-ca56cf0ef2be"
Sep 18 18:45:17.519701 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.519657    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-72pdf"]
Sep 18 18:45:17.540686 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.540642    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.547729 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.547709    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 18:45:17.548235 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.548216    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 18:45:17.549735 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.549717    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zff62\""
Sep 18 18:45:17.555324 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.555304    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-72pdf"]
Sep 18 18:45:17.674867 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.674785    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-data-volume\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.674867 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.674844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-crio-socket\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.675036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.674928    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.675036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.674969    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.675036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.675012    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5t2p\" (UniqueName: \"kubernetes.io/projected/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-api-access-v5t2p\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.776304 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776269    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.776478 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.776478 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776385    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:17.776478 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776430    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:17.776478 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776465    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776487    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:18.276469465 +0000 UTC m=+47.714729651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776547    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:18.27650576 +0000 UTC m=+47.714765955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776576    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776580    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5t2p\" (UniqueName: \"kubernetes.io/projected/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-api-access-v5t2p\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776633    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.776626393 +0000 UTC m=+55.214886575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : secret "router-metrics-certs-default" not found
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776705    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776721    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbd59bc79-h9mc7: secret "image-registry-tls" not found
Sep 18 18:45:17.776751 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776751    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls podName:4cb38fe2-b1e1-47a5-ae73-504df175336e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.776741369 +0000 UTC m=+55.215001554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls") pod "image-registry-7cbd59bc79-h9mc7" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e") : secret "image-registry-tls" not found
Sep 18 18:45:17.777072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776709    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:17.777072 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:17.776774    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.776767459 +0000 UTC m=+55.215027640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : configmap references non-existent config key: service-ca.crt
Sep 18 18:45:17.777072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776822    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-data-volume\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.777072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776846    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-crio-socket\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.777072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.776994    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-crio-socket\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.777216 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.777117    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-data-volume\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:17.788555 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:17.788503    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5t2p\" (UniqueName: \"kubernetes.io/projected/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-api-access-v5t2p\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:18.281535 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:18.281476    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:18.281714 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:18.281545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:18.281714 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:18.281635    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:18.281714 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:18.281648    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:18.281714 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:18.281698    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:19.281685282 +0000 UTC m=+48.719945463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:18.281714 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:18.281712    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:19.281706814 +0000 UTC m=+48.719966995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:18.322253 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:18.322228    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:45:18.322613 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:18.322558    2568 scope.go:117] "RemoveContainer" containerID="2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e"
Sep 18 18:45:18.322715 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:18.322698    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-dncfj_openshift-console-operator(3125a1e7-a100-4427-b812-ca56cf0ef2be)\"" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" podUID="3125a1e7-a100-4427-b812-ca56cf0ef2be"
Sep 18 18:45:19.290873 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:19.290832    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:19.291066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:19.290889    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:19.291066 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:19.290967    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:19.291066 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:19.291002    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:19.291066 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:19.291030    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:21.291013437 +0000 UTC m=+50.729273624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:19.291066 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:19.291066    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:21.291049409 +0000 UTC m=+50.729309606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:20.360965 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:20.360934    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:20.361301 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:20.361186    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:20.361301 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:20.361248    2568 scope.go:117] "RemoveContainer" containerID="2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e"
Sep 18 18:45:20.361396 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:20.361379    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-dncfj_openshift-console-operator(3125a1e7-a100-4427-b812-ca56cf0ef2be)\"" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" podUID="3125a1e7-a100-4427-b812-ca56cf0ef2be"
Sep 18 18:45:20.432831 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:20.432808    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nsfwm_e8386b94-b143-4f08-a95a-ce6fb840c8f3/dns-node-resolver/0.log"
Sep 18 18:45:21.207793 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:21.207760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:21.207938 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.207911    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:45:21.207984 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.207974    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls podName:a48b6c68-2dc3-4f03-b60a-1b181429c33e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:37.20795829 +0000 UTC m=+66.646218471 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls") pod "dns-default-qgl2s" (UID: "a48b6c68-2dc3-4f03-b60a-1b181429c33e") : secret "dns-default-metrics-tls" not found
Sep 18 18:45:21.309109 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:21.309076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:21.309274 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:21.309117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:21.309274 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.309221    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:21.309274 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.309227    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:21.309274 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.309272    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.309258981 +0000 UTC m=+54.747519162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:21.309410 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.309284    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:25.309278749 +0000 UTC m=+54.747538930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:21.330066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:21.330047    2568 scope.go:117] "RemoveContainer" containerID="2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e"
Sep 18 18:45:21.330197 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:21.330181    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-dncfj_openshift-console-operator(3125a1e7-a100-4427-b812-ca56cf0ef2be)\"" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" podUID="3125a1e7-a100-4427-b812-ca56cf0ef2be"
Sep 18 18:45:21.433403 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:21.433374    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vdgtt_1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4/node-ca/0.log"
Sep 18 18:45:22.032779 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:22.032744    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfx72_33113daa-1aea-42d9-82bd-ecaa8d22cc9a/serve-healthcheck-canary/0.log"
Sep 18 18:45:25.342145 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:25.342116    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:25.342653 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:25.342156    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:25.342653 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.342246    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:45:25.342653 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.342278    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:25.342653 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.342306    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:33.342292344 +0000 UTC m=+62.780552526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : configmap "kube-rbac-proxy" not found
Sep 18 18:45:25.342653 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.342353    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:33.342335419 +0000 UTC m=+62.780595600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:25.845466 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:25.845428    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:25.845466 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:25.845472    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:25.845488    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.845596    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.845608    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.845629    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbd59bc79-h9mc7: secret "image-registry-tls" not found
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.845620    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:41.845604572 +0000 UTC m=+71.283864753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : configmap references non-existent config key: service-ca.crt
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.845697    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs podName:85c147de-2421-41c8-b625-33d6a9cb1e58 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:41.845679491 +0000 UTC m=+71.283939675 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs") pod "router-default-8587f4c686-59cps" (UID: "85c147de-2421-41c8-b625-33d6a9cb1e58") : secret "router-metrics-certs-default" not found
Sep 18 18:45:25.845732 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:25.845712    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls podName:4cb38fe2-b1e1-47a5-ae73-504df175336e nodeName:}" failed. No retries permitted until 2026-09-18 18:45:41.845706185 +0000 UTC m=+71.283966366 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls") pod "image-registry-7cbd59bc79-h9mc7" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e") : secret "image-registry-tls" not found
Sep 18 18:45:29.253230 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:29.253195    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85dvq"
Sep 18 18:45:32.048772 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:32.048741    2568 scope.go:117] "RemoveContainer" containerID="2b40e4eb7003ed1c0be47fd1698aba9959e16d497017c04cdaca2701baed7e1e"
Sep 18 18:45:32.360150 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:32.360077    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:45:32.360150 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:32.360139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" event={"ID":"3125a1e7-a100-4427-b812-ca56cf0ef2be","Type":"ContainerStarted","Data":"8ab4e76890de3baea58b6934f3db0c54b2406d5b4a74c60792297247dc5c86d9"}
Sep 18 18:45:32.360443 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:32.360423    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:32.828731 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:32.828697    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj"
Sep 18 18:45:32.853930 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:32.853882    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-dncfj" podStartSLOduration=18.616967935 podStartE2EDuration="23.85386596s" totalImagesPullingTime="5.236898025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:32.39353188 +0000 UTC m=+61.831792077" watchObservedRunningTime="2026-09-18 18:45:32.85386596 +0000 UTC m=+62.292126162"
Sep 18 18:45:33.406116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:33.406087    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:33.406116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:33.406128    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:33.406623 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:33.406229    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:33.406623 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:33.406301    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls podName:b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939 nodeName:}" failed. No retries permitted until 2026-09-18 18:45:49.406276282 +0000 UTC m=+78.844536463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls") pod "insights-runtime-extractor-72pdf" (UID: "b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:45:33.406702 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:33.406653    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:36.729727 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.729688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:36.732110 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.732092    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:45:36.742705 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.742679    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd-original-pull-secret\") pod \"global-pull-secret-syncer-hq4lf\" (UID: \"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd\") " pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:36.764955 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.764922    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hq4lf"
Sep 18 18:45:36.830195 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.830169    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:36.830327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.830211    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:36.832756 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.832714    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4dqh\" (UniqueName: \"kubernetes.io/projected/f3e567a1-85ce-4000-908f-eb34b464ae1f-kube-api-access-p4dqh\") pod \"network-check-target-k5248\" (UID: \"f3e567a1-85ce-4000-908f-eb34b464ae1f\") " pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:36.832960 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.832945    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:45:36.842384 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.842364    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe46cc04-7e73-4bcf-8a60-b684fb221935-metrics-certs\") pod \"network-metrics-daemon-k77tr\" (UID: \"fe46cc04-7e73-4bcf-8a60-b684fb221935\") " pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:36.960376 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.960351    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vk59r\""
Sep 18 18:45:36.968219 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:36.968195    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:37.037509 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.037482    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hq4lf"]
Sep 18 18:45:37.040550 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:37.040527    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fa9a7e_c852_4d5b_9fc4_6d75203eb0dd.slice/crio-cf160bc267b8c2461570985ad22738440ba68748d9db2625ef0bf1e2e99d2000 WatchSource:0}: Error finding container cf160bc267b8c2461570985ad22738440ba68748d9db2625ef0bf1e2e99d2000: Status 404 returned error can't find the container with id cf160bc267b8c2461570985ad22738440ba68748d9db2625ef0bf1e2e99d2000
Sep 18 18:45:37.061631 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.061613    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xdsv4\""
Sep 18 18:45:37.069805 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.069790    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k77tr"
Sep 18 18:45:37.232661 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.232627    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k5248"]
Sep 18 18:45:37.234262 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.234236    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:37.235496 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:37.235468    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e567a1_85ce_4000_908f_eb34b464ae1f.slice/crio-da1228e7bb041dc917caa4af9d0632f01b2015b496527a51c6ce2c45883deb39 WatchSource:0}: Error finding container da1228e7bb041dc917caa4af9d0632f01b2015b496527a51c6ce2c45883deb39: Status 404 returned error can't find the container with id da1228e7bb041dc917caa4af9d0632f01b2015b496527a51c6ce2c45883deb39
Sep 18 18:45:37.236560 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.236538    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a48b6c68-2dc3-4f03-b60a-1b181429c33e-metrics-tls\") pod \"dns-default-qgl2s\" (UID: \"a48b6c68-2dc3-4f03-b60a-1b181429c33e\") " pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:37.337817 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.337794    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k77tr"]
Sep 18 18:45:37.340523 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:37.340486    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe46cc04_7e73_4bcf_8a60_b684fb221935.slice/crio-06769d827ddaa58d4bafcc590a1fbb20c8cf536673807c064558f62b2103dabe WatchSource:0}: Error finding container 06769d827ddaa58d4bafcc590a1fbb20c8cf536673807c064558f62b2103dabe: Status 404 returned error can't find the container with id 06769d827ddaa58d4bafcc590a1fbb20c8cf536673807c064558f62b2103dabe
Sep 18 18:45:37.379403 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.379340    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k77tr" event={"ID":"fe46cc04-7e73-4bcf-8a60-b684fb221935","Type":"ContainerStarted","Data":"06769d827ddaa58d4bafcc590a1fbb20c8cf536673807c064558f62b2103dabe"}
Sep 18 18:45:37.381739 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.381717    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k5248" event={"ID":"f3e567a1-85ce-4000-908f-eb34b464ae1f","Type":"ContainerStarted","Data":"1a8f7c08bd2b2ed46b3da9621b00fb2fb5617ba9af84da443087c66686642dc5"}
Sep 18 18:45:37.381826 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.381752    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k5248" event={"ID":"f3e567a1-85ce-4000-908f-eb34b464ae1f","Type":"ContainerStarted","Data":"da1228e7bb041dc917caa4af9d0632f01b2015b496527a51c6ce2c45883deb39"}
Sep 18 18:45:37.381826 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.381795    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:45:37.382843 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.382823    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hq4lf" event={"ID":"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd","Type":"ContainerStarted","Data":"cf160bc267b8c2461570985ad22738440ba68748d9db2625ef0bf1e2e99d2000"}
Sep 18 18:45:37.400696 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.400660    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k5248" podStartSLOduration=66.400647643 podStartE2EDuration="1m6.400647643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:37.399704917 +0000 UTC m=+66.837965119" watchObservedRunningTime="2026-09-18 18:45:37.400647643 +0000 UTC m=+66.838907837"
Sep 18 18:45:37.464024 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.464007    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qf4nw\""
Sep 18 18:45:37.471575 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.471557    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:37.744404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:37.744369    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qgl2s"]
Sep 18 18:45:37.748625 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:37.748596    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48b6c68_2dc3_4f03_b60a_1b181429c33e.slice/crio-da702fa7424dc0f059d65031c3601866be76d57964a7648c4b524deef3857a54 WatchSource:0}: Error finding container da702fa7424dc0f059d65031c3601866be76d57964a7648c4b524deef3857a54: Status 404 returned error can't find the container with id da702fa7424dc0f059d65031c3601866be76d57964a7648c4b524deef3857a54
Sep 18 18:45:38.387930 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:38.387893    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qgl2s" event={"ID":"a48b6c68-2dc3-4f03-b60a-1b181429c33e","Type":"ContainerStarted","Data":"da702fa7424dc0f059d65031c3601866be76d57964a7648c4b524deef3857a54"}
Sep 18 18:45:39.395268 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:39.395227    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k77tr" event={"ID":"fe46cc04-7e73-4bcf-8a60-b684fb221935","Type":"ContainerStarted","Data":"af7507ec842da48779cf0eecea9e2268720df8aa7849422946c215cc0dcb5d13"}
Sep 18 18:45:39.395739 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:39.395291    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k77tr" event={"ID":"fe46cc04-7e73-4bcf-8a60-b684fb221935","Type":"ContainerStarted","Data":"2f6483343167fd7fd9fb9a7b119ee771d3bd8f229c1031581c77a811fe52ca38"}
Sep 18 18:45:39.420906 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:39.420848    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k77tr" podStartSLOduration=67.101024173 podStartE2EDuration="1m8.420828341s" totalImagesPullingTime="1.319804168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:39.419814411 +0000 UTC m=+68.858074616" watchObservedRunningTime="2026-09-18 18:45:39.420828341 +0000 UTC m=+68.859088545"
Sep 18 18:45:41.874773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:41.874733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:41.875129 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:41.874816    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:41.875129 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:41.874836    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:41.875368 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:41.875345    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85c147de-2421-41c8-b625-33d6a9cb1e58-service-ca-bundle\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:41.877101 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:41.877079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85c147de-2421-41c8-b625-33d6a9cb1e58-metrics-certs\") pod \"router-default-8587f4c686-59cps\" (UID: \"85c147de-2421-41c8-b625-33d6a9cb1e58\") " pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:41.877101 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:41.877094    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"image-registry-7cbd59bc79-h9mc7\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") " pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:42.121310 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.121276    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m6jx4\""
Sep 18 18:45:42.130070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.130019    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:42.139687 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.139637    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2zbnj\""
Sep 18 18:45:42.147459 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.147439    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:42.400757 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.400728    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbd59bc79-h9mc7"]
Sep 18 18:45:42.402939 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:42.402910    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb38fe2_b1e1_47a5_ae73_504df175336e.slice/crio-32dc2efcbdfb8cb557ecfba49f603e1e47dbeea42531bdfad067ab8e9b15b383 WatchSource:0}: Error finding container 32dc2efcbdfb8cb557ecfba49f603e1e47dbeea42531bdfad067ab8e9b15b383: Status 404 returned error can't find the container with id 32dc2efcbdfb8cb557ecfba49f603e1e47dbeea42531bdfad067ab8e9b15b383
Sep 18 18:45:42.406348 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.406317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hq4lf" event={"ID":"51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd","Type":"ContainerStarted","Data":"024a1f748a075c3311e6507159ed49e4a41a4c9781fb665621bd82ac7d507bda"}
Sep 18 18:45:42.408053 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.408030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qgl2s" event={"ID":"a48b6c68-2dc3-4f03-b60a-1b181429c33e","Type":"ContainerStarted","Data":"8cb86220e19eb9c93be1869efa06fa82490111f8935348f425f43ce269d68c22"}
Sep 18 18:45:42.408126 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.408076    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qgl2s" event={"ID":"a48b6c68-2dc3-4f03-b60a-1b181429c33e","Type":"ContainerStarted","Data":"52d60a6ea5ebb3a6a1db6131bcb2823cf03e3060e01b75455c8b39ddec859a38"}
Sep 18 18:45:42.409785 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.409755    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:42.425246 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.425218    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8587f4c686-59cps"]
Sep 18 18:45:42.429145 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.429104    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qgl2s" podStartSLOduration=33.734029829 podStartE2EDuration="37.429089219s" totalImagesPullingTime="3.69505939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:42.428019457 +0000 UTC m=+71.866279661" watchObservedRunningTime="2026-09-18 18:45:42.429089219 +0000 UTC m=+71.867349422"
Sep 18 18:45:42.434501 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:42.434475    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c147de_2421_41c8_b625_33d6a9cb1e58.slice/crio-705b0d34150a91586f09df6d34927e9498bc21f4e79fb4c4f7b7e0e6a92d97b2 WatchSource:0}: Error finding container 705b0d34150a91586f09df6d34927e9498bc21f4e79fb4c4f7b7e0e6a92d97b2: Status 404 returned error can't find the container with id 705b0d34150a91586f09df6d34927e9498bc21f4e79fb4c4f7b7e0e6a92d97b2
Sep 18 18:45:42.445096 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:42.445054    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hq4lf" podStartSLOduration=67.035943292 podStartE2EDuration="1m11.445038891s" totalImagesPullingTime="4.409095599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:42.44419742 +0000 UTC m=+71.882457624" watchObservedRunningTime="2026-09-18 18:45:42.445038891 +0000 UTC m=+71.883299089"
Sep 18 18:45:43.412490 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.412451    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8587f4c686-59cps" event={"ID":"85c147de-2421-41c8-b625-33d6a9cb1e58","Type":"ContainerStarted","Data":"ea5fddc4655b8ea3adff3e01a924255ec10993f84a64500529abdfd11d413aae"}
Sep 18 18:45:43.412875 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.412508    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8587f4c686-59cps" event={"ID":"85c147de-2421-41c8-b625-33d6a9cb1e58","Type":"ContainerStarted","Data":"705b0d34150a91586f09df6d34927e9498bc21f4e79fb4c4f7b7e0e6a92d97b2"}
Sep 18 18:45:43.414002 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.413975    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7" event={"ID":"4cb38fe2-b1e1-47a5-ae73-504df175336e","Type":"ContainerStarted","Data":"1eb06422c1cd7ea01eb90f82a1b0be3c0a305bc29155fe5446f10e259c90e8ad"}
Sep 18 18:45:43.414124 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.414015    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7" event={"ID":"4cb38fe2-b1e1-47a5-ae73-504df175336e","Type":"ContainerStarted","Data":"32dc2efcbdfb8cb557ecfba49f603e1e47dbeea42531bdfad067ab8e9b15b383"}
Sep 18 18:45:43.414124 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.414066    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:45:43.432651 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.432602    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-8587f4c686-59cps" podStartSLOduration=34.432586449 podStartE2EDuration="34.432586449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:43.432183279 +0000 UTC m=+72.870443485" watchObservedRunningTime="2026-09-18 18:45:43.432586449 +0000 UTC m=+72.870846653"
Sep 18 18:45:43.455187 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.455142    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7" podStartSLOduration=34.45512989 podStartE2EDuration="34.45512989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:43.454332278 +0000 UTC m=+72.892592481" watchObservedRunningTime="2026-09-18 18:45:43.45512989 +0000 UTC m=+72.893390092"
Sep 18 18:45:43.961387 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:43.961353    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbd59bc79-h9mc7"]
Sep 18 18:45:44.012242 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.012212    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-h4xb8"]
Sep 18 18:45:44.017011 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.016990    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fcf6db547-s4hrj"]
Sep 18 18:45:44.017129 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.017113    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:45:44.019817 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.019787    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.019919 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.019861    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bgrsc\""
Sep 18 18:45:44.019967 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.019925    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 18:45:44.020005 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.019975    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 18:45:44.028699 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.028679    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-h4xb8"]
Sep 18 18:45:44.041382 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.039546    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fcf6db547-s4hrj"]
Sep 18 18:45:44.091568 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091543    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-bound-sa-token\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091686 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091592    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx9kk\" (UniqueName: \"kubernetes.io/projected/ac032255-b81a-41bd-9a08-278e796f31c7-kube-api-access-zx9kk\") pod \"downloads-6594f4c75-h4xb8\" (UID: \"ac032255-b81a-41bd-9a08-278e796f31c7\") " pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:45:44.091686 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f69e02dd-39f8-4394-8e29-63bf680f4817-image-registry-private-configuration\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091686 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f69e02dd-39f8-4394-8e29-63bf680f4817-installation-pull-secrets\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091686 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091660    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glw7c\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-kube-api-access-glw7c\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091686 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091675    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f69e02dd-39f8-4394-8e29-63bf680f4817-trusted-ca\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091835 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091698    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f69e02dd-39f8-4394-8e29-63bf680f4817-ca-trust-extracted\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091835 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f69e02dd-39f8-4394-8e29-63bf680f4817-registry-certificates\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.091897 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.091839    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-registry-tls\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.147755 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.147706    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:44.150135 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.150112    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:44.192349 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx9kk\" (UniqueName: \"kubernetes.io/projected/ac032255-b81a-41bd-9a08-278e796f31c7-kube-api-access-zx9kk\") pod \"downloads-6594f4c75-h4xb8\" (UID: \"ac032255-b81a-41bd-9a08-278e796f31c7\") " pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:45:44.192349 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192351    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f69e02dd-39f8-4394-8e29-63bf680f4817-image-registry-private-configuration\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192580 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192381    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f69e02dd-39f8-4394-8e29-63bf680f4817-installation-pull-secrets\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192633 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glw7c\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-kube-api-access-glw7c\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192680 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192631    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f69e02dd-39f8-4394-8e29-63bf680f4817-trusted-ca\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192735 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192684    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f69e02dd-39f8-4394-8e29-63bf680f4817-ca-trust-extracted\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192832 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192806    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f69e02dd-39f8-4394-8e29-63bf680f4817-registry-certificates\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192970 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192856    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-registry-tls\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.192970 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.192889    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-bound-sa-token\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.193078 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.193064    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f69e02dd-39f8-4394-8e29-63bf680f4817-ca-trust-extracted\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.193481 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.193461    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f69e02dd-39f8-4394-8e29-63bf680f4817-trusted-ca\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.193609 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.193587    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f69e02dd-39f8-4394-8e29-63bf680f4817-registry-certificates\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.194810 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.194791    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f69e02dd-39f8-4394-8e29-63bf680f4817-installation-pull-secrets\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.194975 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.194959    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f69e02dd-39f8-4394-8e29-63bf680f4817-image-registry-private-configuration\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.195171 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.195148    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-registry-tls\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.209497 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.209467    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-bound-sa-token\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.209834 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.209810    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx9kk\" (UniqueName: \"kubernetes.io/projected/ac032255-b81a-41bd-9a08-278e796f31c7-kube-api-access-zx9kk\") pod \"downloads-6594f4c75-h4xb8\" (UID: \"ac032255-b81a-41bd-9a08-278e796f31c7\") " pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:45:44.209874 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.209846    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glw7c\" (UniqueName: \"kubernetes.io/projected/f69e02dd-39f8-4394-8e29-63bf680f4817-kube-api-access-glw7c\") pod \"image-registry-fcf6db547-s4hrj\" (UID: \"f69e02dd-39f8-4394-8e29-63bf680f4817\") " pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.326448 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.326410    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:45:44.332665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.332642    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:44.418256 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.418224    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:44.419361 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.419338    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8587f4c686-59cps"
Sep 18 18:45:44.605104 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.605084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-h4xb8"]
Sep 18 18:45:44.607230 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:44.607203    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac032255_b81a_41bd_9a08_278e796f31c7.slice/crio-70a25f78bbed5e205cad53fe9d8fb94b7c649f71c6da56ed588ebea15c64ee64 WatchSource:0}: Error finding container 70a25f78bbed5e205cad53fe9d8fb94b7c649f71c6da56ed588ebea15c64ee64: Status 404 returned error can't find the container with id 70a25f78bbed5e205cad53fe9d8fb94b7c649f71c6da56ed588ebea15c64ee64
Sep 18 18:45:44.610791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:44.610773    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fcf6db547-s4hrj"]
Sep 18 18:45:44.612914 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:44.612892    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69e02dd_39f8_4394_8e29_63bf680f4817.slice/crio-c1c4a3f7d5ee825fa1ceff9304eece0ae439107ee4c1ee49d6341535743187f5 WatchSource:0}: Error finding container c1c4a3f7d5ee825fa1ceff9304eece0ae439107ee4c1ee49d6341535743187f5: Status 404 returned error can't find the container with id c1c4a3f7d5ee825fa1ceff9304eece0ae439107ee4c1ee49d6341535743187f5
Sep 18 18:45:45.422431 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:45.422367    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-h4xb8" event={"ID":"ac032255-b81a-41bd-9a08-278e796f31c7","Type":"ContainerStarted","Data":"70a25f78bbed5e205cad53fe9d8fb94b7c649f71c6da56ed588ebea15c64ee64"}
Sep 18 18:45:45.424209 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:45.424172    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj" event={"ID":"f69e02dd-39f8-4394-8e29-63bf680f4817","Type":"ContainerStarted","Data":"28f900b729d3192a4ef12d829e5e1b357adc3802096cb6b0336591ee6600ac3c"}
Sep 18 18:45:45.424346 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:45.424230    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj" event={"ID":"f69e02dd-39f8-4394-8e29-63bf680f4817","Type":"ContainerStarted","Data":"c1c4a3f7d5ee825fa1ceff9304eece0ae439107ee4c1ee49d6341535743187f5"}
Sep 18 18:45:45.472965 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:45.472919    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj" podStartSLOduration=2.472903836 podStartE2EDuration="2.472903836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:45.471876534 +0000 UTC m=+74.910136737" watchObservedRunningTime="2026-09-18 18:45:45.472903836 +0000 UTC m=+74.911164039"
Sep 18 18:45:46.428487 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:46.428426    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:45:49.437860 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:49.437815    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:49.440670 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:49.440642    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-72pdf\" (UID: \"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939\") " pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:49.651232 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:49.651194    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zff62\""
Sep 18 18:45:49.659243 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:49.659217    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-72pdf"
Sep 18 18:45:49.967706 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:49.967664    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-72pdf"]
Sep 18 18:45:49.971111 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:45:49.971083    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39bfa58_d06f_4ad4_bf2e_3cd93d3aa939.slice/crio-4abdf9d7d9b61d6727c4ffd9f7e555cc7f6c84671ebfdd267bed2dac62967bd7 WatchSource:0}: Error finding container 4abdf9d7d9b61d6727c4ffd9f7e555cc7f6c84671ebfdd267bed2dac62967bd7: Status 404 returned error can't find the container with id 4abdf9d7d9b61d6727c4ffd9f7e555cc7f6c84671ebfdd267bed2dac62967bd7
Sep 18 18:45:50.446797 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:50.446719    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-72pdf" event={"ID":"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939","Type":"ContainerStarted","Data":"44f471fb8eb91f96dea817b17fc5b1a11729b415f1cd28d9a285c203933dcd47"}
Sep 18 18:45:50.446797 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:50.446786    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-72pdf" event={"ID":"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939","Type":"ContainerStarted","Data":"4abdf9d7d9b61d6727c4ffd9f7e555cc7f6c84671ebfdd267bed2dac62967bd7"}
Sep 18 18:45:51.454836 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:51.454792    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-72pdf" event={"ID":"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939","Type":"ContainerStarted","Data":"202b6efc0957d1245eb284fe61f7e60a3bcbf285600c61bff28e266acd793358"}
Sep 18 18:45:52.417201 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:52.417171    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qgl2s"
Sep 18 18:45:53.468960 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:53.468919    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-72pdf" event={"ID":"b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939","Type":"ContainerStarted","Data":"57b00903e4af1578be4380e400d1e8a8bc282b5265aec60edc295345e00d19f3"}
Sep 18 18:45:53.500050 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:53.499915    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-72pdf" podStartSLOduration=34.08162443 podStartE2EDuration="36.499893274s" totalImagesPullingTime="2.418268844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:45:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:45:53.490967725 +0000 UTC m=+82.929227942" watchObservedRunningTime="2026-09-18 18:45:53.499893274 +0000 UTC m=+82.938153478"
Sep 18 18:45:54.121335 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.121296    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"]
Sep 18 18:45:54.124152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.124127    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.126418 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.126381    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 18:45:54.126566 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.126425    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 18:45:54.127098 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.127072    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 18:45:54.127248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.127078    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qrl6d\""
Sep 18 18:45:54.127248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.127182    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 18:45:54.127248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.127232    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 18:45:54.134370 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.134348    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"]
Sep 18 18:45:54.177745 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.177710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.177875 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.177759    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f63723ad-8bbf-4b77-8e17-775f4c503e7b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.177929 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.177870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.177929 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.177907    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r44k4\" (UniqueName: \"kubernetes.io/projected/f63723ad-8bbf-4b77-8e17-775f4c503e7b-kube-api-access-r44k4\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.279121 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.279081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.279311 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.279135    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f63723ad-8bbf-4b77-8e17-775f4c503e7b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.279311 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.279204    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.279311 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.279232    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r44k4\" (UniqueName: \"kubernetes.io/projected/f63723ad-8bbf-4b77-8e17-775f4c503e7b-kube-api-access-r44k4\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.279465 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:54.279327    2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 18 18:45:54.279465 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:45:54.279405    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-tls podName:f63723ad-8bbf-4b77-8e17-775f4c503e7b nodeName:}" failed. No retries permitted until 2026-09-18 18:45:54.779381996 +0000 UTC m=+84.217642185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-zlwsf" (UID: "f63723ad-8bbf-4b77-8e17-775f4c503e7b") : secret "prometheus-operator-tls" not found
Sep 18 18:45:54.280047 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.280026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f63723ad-8bbf-4b77-8e17-775f4c503e7b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.282049 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.282028    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.288214 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.288195    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r44k4\" (UniqueName: \"kubernetes.io/projected/f63723ad-8bbf-4b77-8e17-775f4c503e7b-kube-api-access-r44k4\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.783872 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.783836    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:54.786428 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:54.786396    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f63723ad-8bbf-4b77-8e17-775f4c503e7b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zlwsf\" (UID: \"f63723ad-8bbf-4b77-8e17-775f4c503e7b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:45:55.035419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:45:55.035335    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"
Sep 18 18:46:00.421640 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:00.421561    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf"]
Sep 18 18:46:00.425347 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:46:00.425308    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63723ad_8bbf_4b77_8e17_775f4c503e7b.slice/crio-9b0620c2e556a13dec62cbed84b79e9368273f88b508c2cf74d2aaa86a00bb40 WatchSource:0}: Error finding container 9b0620c2e556a13dec62cbed84b79e9368273f88b508c2cf74d2aaa86a00bb40: Status 404 returned error can't find the container with id 9b0620c2e556a13dec62cbed84b79e9368273f88b508c2cf74d2aaa86a00bb40
Sep 18 18:46:00.496893 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:00.496850    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf" event={"ID":"f63723ad-8bbf-4b77-8e17-775f4c503e7b","Type":"ContainerStarted","Data":"9b0620c2e556a13dec62cbed84b79e9368273f88b508c2cf74d2aaa86a00bb40"}
Sep 18 18:46:00.498836 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:00.498803    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-h4xb8" event={"ID":"ac032255-b81a-41bd-9a08-278e796f31c7","Type":"ContainerStarted","Data":"2911ae325f68ab73aaf0a856ffe077b125034d41e09c4258ec39e3792ada529e"}
Sep 18 18:46:00.499121 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:00.499103    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:46:00.515919 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:00.515893    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-h4xb8"
Sep 18 18:46:00.520405 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:00.520160    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-h4xb8" podStartSLOduration=1.900430654 podStartE2EDuration="17.520143154s" totalImagesPullingTime="15.6197125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:00.520061204 +0000 UTC m=+89.958321409" watchObservedRunningTime="2026-09-18 18:46:00.520143154 +0000 UTC m=+89.958403415"
Sep 18 18:46:02.510057 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:02.510020    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf" event={"ID":"f63723ad-8bbf-4b77-8e17-775f4c503e7b","Type":"ContainerStarted","Data":"07fc5b4e7b533c0372ad853c8242f1800a793b58ad5739604c5bc7060ebe4052"}
Sep 18 18:46:03.516566 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:03.516510    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf" event={"ID":"f63723ad-8bbf-4b77-8e17-775f4c503e7b","Type":"ContainerStarted","Data":"8ec1a577063c2918bf14962ee85ecd846e89b068b47c3c9ce5cc9c64d14b4f83"}
Sep 18 18:46:03.533684 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:03.533636    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zlwsf" podStartSLOduration=7.687354519 podStartE2EDuration="9.533620954s" totalImagesPullingTime="1.846266435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:45:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:03.532630137 +0000 UTC m=+92.970890352" watchObservedRunningTime="2026-09-18 18:46:03.533620954 +0000 UTC m=+92.971881177"
Sep 18 18:46:05.431250 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.431215    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:46:05.583496 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.583454    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"]
Sep 18 18:46:05.607116 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.607082    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"]
Sep 18 18:46:05.607294 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.607119    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.609815 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.609784    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 18:46:05.609961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.609820    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 18:46:05.609961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.609867    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2g6dp\""
Sep 18 18:46:05.624649 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.624618    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"]
Sep 18 18:46:05.639376 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.639344    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mwzgb"]
Sep 18 18:46:05.639556 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.639485    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.642250 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.642218    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rkdql\""
Sep 18 18:46:05.642394 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.642375    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 18 18:46:05.642394 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.642388    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 18 18:46:05.642503 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.642460    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 18 18:46:05.655706 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.655680    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.660918 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.660883    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-js7p5\""
Sep 18 18:46:05.660918 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.660883    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 18:46:05.661122 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.660928    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 18:46:05.661122 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.660997    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 18:46:05.670317 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.670292    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"]
Sep 18 18:46:05.779879 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.779838    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.780075 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.779903    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.780075 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.779936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780075 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.779972    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2mtf\" (UniqueName: \"kubernetes.io/projected/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-api-access-j2mtf\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.780075 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.779995    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-tls\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780299 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780078    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp8dv\" (UniqueName: \"kubernetes.io/projected/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-kube-api-access-mp8dv\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780299 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780140    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.780299 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780190    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9fc2\" (UniqueName: \"kubernetes.io/projected/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-kube-api-access-c9fc2\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.780299 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780260    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-metrics-client-ca\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780299 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780312    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780338    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-textfile\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-wtmp\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780398    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-sys\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780431    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780458    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.780569 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780555    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.780866 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.780608    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-root\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.881561 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-textfile\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881568    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-wtmp\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-sys\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881629    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881689    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881720    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-sys\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.881768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881737    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-root\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881771    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-wtmp\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881783    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-root\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881790    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881843    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2mtf\" (UniqueName: \"kubernetes.io/projected/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-api-access-j2mtf\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881924    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-tls\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881950    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp8dv\" (UniqueName: \"kubernetes.io/projected/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-kube-api-access-mp8dv\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881982    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882010    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9fc2\" (UniqueName: \"kubernetes.io/projected/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-kube-api-access-c9fc2\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-metrics-client-ca\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882091    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.882119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882125    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.882758 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882460    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.882758 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882463    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882758 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882542    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.882758 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882647    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.882758 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.881946    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-textfile\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.882980 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.882894    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.883578 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.883528    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-metrics-client-ca\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.885075 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.885002    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.885192 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.885095    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.885743 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.885715    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.885854 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.885831    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-node-exporter-tls\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.886791 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.886768    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.887051 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.887026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.892150 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.892127    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp8dv\" (UniqueName: \"kubernetes.io/projected/54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b-kube-api-access-mp8dv\") pod \"node-exporter-mwzgb\" (UID: \"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b\") " pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:05.892240 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.892129    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2mtf\" (UniqueName: \"kubernetes.io/projected/54c8cb6a-e9d6-4e73-bfa6-06c03557bed9-kube-api-access-j2mtf\") pod \"kube-state-metrics-84c885658c-xh4mn\" (UID: \"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.892618 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.892595    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9fc2\" (UniqueName: \"kubernetes.io/projected/2f6632a5-2f4f-49b0-95c7-a84ff5480f76-kube-api-access-c9fc2\") pod \"openshift-state-metrics-86c5bc79d6-pxq2x\" (UID: \"2f6632a5-2f4f-49b0-95c7-a84ff5480f76\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.918339 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.918301    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"
Sep 18 18:46:05.951072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.950552    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"
Sep 18 18:46:05.965505 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:05.965476    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mwzgb"
Sep 18 18:46:06.231551 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.231495    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x"]
Sep 18 18:46:06.234740 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:46:06.234700    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6632a5_2f4f_49b0_95c7_a84ff5480f76.slice/crio-23884b235de495a908a17b659f8c76a217c8a0a43b2fd4d9cd8922da924d539c WatchSource:0}: Error finding container 23884b235de495a908a17b659f8c76a217c8a0a43b2fd4d9cd8922da924d539c: Status 404 returned error can't find the container with id 23884b235de495a908a17b659f8c76a217c8a0a43b2fd4d9cd8922da924d539c
Sep 18 18:46:06.293912 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.293879    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-xh4mn"]
Sep 18 18:46:06.296795 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:46:06.296757    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c8cb6a_e9d6_4e73_bfa6_06c03557bed9.slice/crio-d1c5691c1f1efaacbe957800319d6eeec7b93020eec285557de71805abbf8230 WatchSource:0}: Error finding container d1c5691c1f1efaacbe957800319d6eeec7b93020eec285557de71805abbf8230: Status 404 returned error can't find the container with id d1c5691c1f1efaacbe957800319d6eeec7b93020eec285557de71805abbf8230
Sep 18 18:46:06.535419 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.535129    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwzgb" event={"ID":"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b","Type":"ContainerStarted","Data":"7860d981ba7ab25cb4ef09e3dd01b06705cf3bc48bcaef622f56881749e9fe0a"}
Sep 18 18:46:06.536712 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.536685    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn" event={"ID":"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9","Type":"ContainerStarted","Data":"d1c5691c1f1efaacbe957800319d6eeec7b93020eec285557de71805abbf8230"}
Sep 18 18:46:06.538751 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.538725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x" event={"ID":"2f6632a5-2f4f-49b0-95c7-a84ff5480f76","Type":"ContainerStarted","Data":"937b1202ea6d70f48848316a5cca1e81ba4e6b64c6b71396eef6e4ba8bdb0840"}
Sep 18 18:46:06.538873 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.538781    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x" event={"ID":"2f6632a5-2f4f-49b0-95c7-a84ff5480f76","Type":"ContainerStarted","Data":"292d847db78f131d84320ea7ea503e5e0dd202732d6d4188572bce9f10733ea6"}
Sep 18 18:46:06.538873 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:06.538808    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x" event={"ID":"2f6632a5-2f4f-49b0-95c7-a84ff5480f76","Type":"ContainerStarted","Data":"23884b235de495a908a17b659f8c76a217c8a0a43b2fd4d9cd8922da924d539c"}
Sep 18 18:46:07.438378 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:07.438332    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fcf6db547-s4hrj"
Sep 18 18:46:08.391625 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:08.391596    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k5248"
Sep 18 18:46:08.552733 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:08.552695    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwzgb" event={"ID":"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b","Type":"ContainerStarted","Data":"f466e751737370602a40d788a11e5eea1bf3a41b58ffd4c646fc5d47261a1c67"}
Sep 18 18:46:09.559115 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.559018    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn" event={"ID":"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9","Type":"ContainerStarted","Data":"6b71e61c1f73335f6fc7a26d25102d3b7e9202396fd8e57230709b9d76debe19"}
Sep 18 18:46:09.559115 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.559102    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn" event={"ID":"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9","Type":"ContainerStarted","Data":"14bfdbbeeab71cf9ac92afbe98862396e0318af9471edfb0e180487553764f07"}
Sep 18 18:46:09.559746 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.559138    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn" event={"ID":"54c8cb6a-e9d6-4e73-bfa6-06c03557bed9","Type":"ContainerStarted","Data":"384b88c4492abe4c44830550992e2f0cf832134189e2a398c620d5a317233310"}
Sep 18 18:46:09.565400 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.565369    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x" event={"ID":"2f6632a5-2f4f-49b0-95c7-a84ff5480f76","Type":"ContainerStarted","Data":"93fbf5843f49df32e4d5b72958037615daf861f91ebebd10750faecbc948aabf"}
Sep 18 18:46:09.566810 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.566786    2568 generic.go:356] "Generic (PLEG): container finished" podID="54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b" containerID="f466e751737370602a40d788a11e5eea1bf3a41b58ffd4c646fc5d47261a1c67" exitCode=0
Sep 18 18:46:09.566948 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.566844    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwzgb" event={"ID":"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b","Type":"ContainerDied","Data":"f466e751737370602a40d788a11e5eea1bf3a41b58ffd4c646fc5d47261a1c67"}
Sep 18 18:46:09.581957 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.581913    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-xh4mn" podStartSLOduration=1.948435222 podStartE2EDuration="4.581896442s" totalImagesPullingTime="2.63346122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:09.580187254 +0000 UTC m=+99.018447456" watchObservedRunningTime="2026-09-18 18:46:09.581896442 +0000 UTC m=+99.020157018"
Sep 18 18:46:09.601491 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:09.601421    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-pxq2x" podStartSLOduration=1.792141655 podStartE2EDuration="4.601401542s" totalImagesPullingTime="2.809259887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:09.598024483 +0000 UTC m=+99.036284687" watchObservedRunningTime="2026-09-18 18:46:09.601401542 +0000 UTC m=+99.039661746"
Sep 18 18:46:10.043392 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.043348    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b4745fdf5-dvks9"]
Sep 18 18:46:10.060007 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.059977    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b4745fdf5-dvks9"]
Sep 18 18:46:10.060156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.060069    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.062744 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.062721    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 18:46:10.063085 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.063062    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 18:46:10.063202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.063159    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 18:46:10.063341 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.063315    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 18:46:10.063445 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.063396    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-40uv5nd1mg09r\""
Sep 18 18:46:10.063504 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.063472    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bm6rb\""
Sep 18 18:46:10.224535 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224491    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcsnr\" (UniqueName: \"kubernetes.io/projected/b51f082b-ac56-416a-8532-ae1c87a60e81-kube-api-access-lcsnr\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.224941 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b51f082b-ac56-416a-8532-ae1c87a60e81-audit-log\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.224941 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224636    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51f082b-ac56-416a-8532-ae1c87a60e81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.224941 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b51f082b-ac56-416a-8532-ae1c87a60e81-metrics-server-audit-profiles\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.224941 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224722    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-secret-metrics-server-client-certs\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.224941 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-client-ca-bundle\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.224941 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.224814    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-secret-metrics-server-tls\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.284776 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.284741    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"]
Sep 18 18:46:10.303710 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.303623    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"]
Sep 18 18:46:10.303847 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.303738    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.307417 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.306828    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8s6cz\""
Sep 18 18:46:10.307602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.307432    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 18 18:46:10.326133 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326104    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-client-ca-bundle\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-secret-metrics-server-tls\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcsnr\" (UniqueName: \"kubernetes.io/projected/b51f082b-ac56-416a-8532-ae1c87a60e81-kube-api-access-lcsnr\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326185    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b51f082b-ac56-416a-8532-ae1c87a60e81-audit-log\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326219    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51f082b-ac56-416a-8532-ae1c87a60e81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326248 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326239    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b51f082b-ac56-416a-8532-ae1c87a60e81-metrics-server-audit-profiles\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326478 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326256    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-secret-metrics-server-client-certs\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.326715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.326663    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b51f082b-ac56-416a-8532-ae1c87a60e81-audit-log\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.327031 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.327005    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b51f082b-ac56-416a-8532-ae1c87a60e81-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.327236 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.327220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b51f082b-ac56-416a-8532-ae1c87a60e81-metrics-server-audit-profiles\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.328751 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.328723    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-secret-metrics-server-tls\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.328828 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.328735    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-secret-metrics-server-client-certs\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.328872 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.328832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b51f082b-ac56-416a-8532-ae1c87a60e81-client-ca-bundle\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.374552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.374498    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcsnr\" (UniqueName: \"kubernetes.io/projected/b51f082b-ac56-416a-8532-ae1c87a60e81-kube-api-access-lcsnr\") pod \"metrics-server-b4745fdf5-dvks9\" (UID: \"b51f082b-ac56-416a-8532-ae1c87a60e81\") " pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.427697 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.427658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bqpn\" (UniqueName: \"kubernetes.io/projected/66b48134-5307-4c8d-a9da-001c9fb0dd32-kube-api-access-9bqpn\") pod \"monitoring-plugin-5984459ff8-d9bn9\" (UID: \"66b48134-5307-4c8d-a9da-001c9fb0dd32\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.427869 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.427712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66b48134-5307-4c8d-a9da-001c9fb0dd32-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d9bn9\" (UID: \"66b48134-5307-4c8d-a9da-001c9fb0dd32\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.444155 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.444106    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7" podUID="4cb38fe2-b1e1-47a5-ae73-504df175336e" containerName="registry" containerID="cri-o://1eb06422c1cd7ea01eb90f82a1b0be3c0a305bc29155fe5446f10e259c90e8ad" gracePeriod=30
Sep 18 18:46:10.528903 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.528864    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bqpn\" (UniqueName: \"kubernetes.io/projected/66b48134-5307-4c8d-a9da-001c9fb0dd32-kube-api-access-9bqpn\") pod \"monitoring-plugin-5984459ff8-d9bn9\" (UID: \"66b48134-5307-4c8d-a9da-001c9fb0dd32\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.529094 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.528920    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66b48134-5307-4c8d-a9da-001c9fb0dd32-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d9bn9\" (UID: \"66b48134-5307-4c8d-a9da-001c9fb0dd32\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.531312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.531289    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66b48134-5307-4c8d-a9da-001c9fb0dd32-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d9bn9\" (UID: \"66b48134-5307-4c8d-a9da-001c9fb0dd32\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.537070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.537045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bqpn\" (UniqueName: \"kubernetes.io/projected/66b48134-5307-4c8d-a9da-001c9fb0dd32-kube-api-access-9bqpn\") pod \"monitoring-plugin-5984459ff8-d9bn9\" (UID: \"66b48134-5307-4c8d-a9da-001c9fb0dd32\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.572922 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.572856    2568 generic.go:356] "Generic (PLEG): container finished" podID="4cb38fe2-b1e1-47a5-ae73-504df175336e" containerID="1eb06422c1cd7ea01eb90f82a1b0be3c0a305bc29155fe5446f10e259c90e8ad" exitCode=0
Sep 18 18:46:10.573283 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.572944    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7" event={"ID":"4cb38fe2-b1e1-47a5-ae73-504df175336e","Type":"ContainerDied","Data":"1eb06422c1cd7ea01eb90f82a1b0be3c0a305bc29155fe5446f10e259c90e8ad"}
Sep 18 18:46:10.575761 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.575728    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwzgb" event={"ID":"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b","Type":"ContainerStarted","Data":"08342cf5369cd498ffa55b46bcbd066488caadc089d7624b4316b7b8e4acd94f"}
Sep 18 18:46:10.575887 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.575786    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwzgb" event={"ID":"54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b","Type":"ContainerStarted","Data":"ae18980cc335c6b717e0b8096dafd8810a3ab03cb555edffa4d5f25bbcfe2927"}
Sep 18 18:46:10.600263 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.600203    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mwzgb" podStartSLOduration=3.200595882 podStartE2EDuration="5.600182066s" totalImagesPullingTime="2.323948584s" totalInitContainerRuntime="75.6376ms" isStatefulPod=true podCreationTimestamp="2026-09-18 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:10.59550611 +0000 UTC m=+100.033766314" watchObservedRunningTime="2026-09-18 18:46:10.600182066 +0000 UTC m=+100.038442269"
Sep 18 18:46:10.614314 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.614282    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:10.670045 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.669832    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:10.726392 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.726354    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:46:10.833610 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833491    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-image-registry-private-configuration\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833610 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833579    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cb38fe2-b1e1-47a5-ae73-504df175336e-ca-trust-extracted\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833610 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833609    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpfnj\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-kube-api-access-fpfnj\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833639    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-certificates\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833663    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-installation-pull-secrets\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833697    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-trusted-ca\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833729    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-bound-sa-token\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.833896 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.833770    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") pod \"4cb38fe2-b1e1-47a5-ae73-504df175336e\" (UID: \"4cb38fe2-b1e1-47a5-ae73-504df175336e\") "
Sep 18 18:46:10.834148 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.834120    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:10.834315 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.834283    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:46:10.836103 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.836071    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:10.836174 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.836150    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:10.836353 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.836319    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:10.836431 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.836402    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-kube-api-access-fpfnj" (OuterVolumeSpecName: "kube-api-access-fpfnj") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "kube-api-access-fpfnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:46:10.836431 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.836413    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:46:10.842151 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.842129    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb38fe2-b1e1-47a5-ae73-504df175336e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4cb38fe2-b1e1-47a5-ae73-504df175336e" (UID: "4cb38fe2-b1e1-47a5-ae73-504df175336e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:46:10.901201 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.901177    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"]
Sep 18 18:46:10.903204 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:46:10.903176    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b48134_5307_4c8d_a9da_001c9fb0dd32.slice/crio-c1c5d3d4d9290983f51743d916723d7fb4edef7cf74ca93869faf4edb7e9c1db WatchSource:0}: Error finding container c1c5d3d4d9290983f51743d916723d7fb4edef7cf74ca93869faf4edb7e9c1db: Status 404 returned error can't find the container with id c1c5d3d4d9290983f51743d916723d7fb4edef7cf74ca93869faf4edb7e9c1db
Sep 18 18:46:10.934933 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934906    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cb38fe2-b1e1-47a5-ae73-504df175336e-ca-trust-extracted\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.934933 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934929    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpfnj\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-kube-api-access-fpfnj\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.934933 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934939    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-certificates\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.935106 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934947    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-installation-pull-secrets\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.935106 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934956    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cb38fe2-b1e1-47a5-ae73-504df175336e-trusted-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.935106 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934964    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-bound-sa-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.935106 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934971    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cb38fe2-b1e1-47a5-ae73-504df175336e-registry-tls\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.935106 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.934979    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cb38fe2-b1e1-47a5-ae73-504df175336e-image-registry-private-configuration\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:46:10.981315 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:10.981284    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b4745fdf5-dvks9"]
Sep 18 18:46:11.580668 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.580636    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7"
Sep 18 18:46:11.581163 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.580636    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbd59bc79-h9mc7" event={"ID":"4cb38fe2-b1e1-47a5-ae73-504df175336e","Type":"ContainerDied","Data":"32dc2efcbdfb8cb557ecfba49f603e1e47dbeea42531bdfad067ab8e9b15b383"}
Sep 18 18:46:11.581163 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.580764    2568 scope.go:117] "RemoveContainer" containerID="1eb06422c1cd7ea01eb90f82a1b0be3c0a305bc29155fe5446f10e259c90e8ad"
Sep 18 18:46:11.582121 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.581927    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9" event={"ID":"b51f082b-ac56-416a-8532-ae1c87a60e81","Type":"ContainerStarted","Data":"8b0965ac00f2cab0b2bfd74e1cc881c924a285e3be9c644fd44f2fec320fcf10"}
Sep 18 18:46:11.583262 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.583235    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9" event={"ID":"66b48134-5307-4c8d-a9da-001c9fb0dd32","Type":"ContainerStarted","Data":"c1c5d3d4d9290983f51743d916723d7fb4edef7cf74ca93869faf4edb7e9c1db"}
Sep 18 18:46:11.603110 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.603081    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbd59bc79-h9mc7"]
Sep 18 18:46:11.606945 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.606924    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cbd59bc79-h9mc7"]
Sep 18 18:46:11.958258 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.958219    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:46:11.958691 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.958670    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb38fe2-b1e1-47a5-ae73-504df175336e" containerName="registry"
Sep 18 18:46:11.958767 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.958694    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb38fe2-b1e1-47a5-ae73-504df175336e" containerName="registry"
Sep 18 18:46:11.958823 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.958798    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cb38fe2-b1e1-47a5-ae73-504df175336e" containerName="registry"
Sep 18 18:46:11.987293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.987203    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:46:11.987472 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.987355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:11.991731 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.991705    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 18:46:11.991898 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.991705    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vq29l\""
Sep 18 18:46:11.992150 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.992132    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 18:46:11.992317 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.992296    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.997851    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.998083    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.998283    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.998724    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.999277    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9n82qkq51vtts\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.999551    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:11.999727    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.000040    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.000218    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 18:46:12.001355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.000424    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 18:46:12.006369 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.006346    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 18:46:12.018602 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.018575    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 18:46:12.152703 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.152959 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.152959 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.152959 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.152959 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152815    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.152959 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152835    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.152959 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152855    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98mfj\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-kube-api-access-98mfj\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.152950    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153010    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-web-config\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153151    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153184    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153206    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153243    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config-out\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153300    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153330    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153363    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.153383 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153395    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.155332 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153427    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.155332 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.153453    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255119 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255078    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255134    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255224    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255246    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255269    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255311    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98mfj\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-kube-api-access-98mfj\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255361    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-web-config\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255399    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255436    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255539    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config-out\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255566    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.255626 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255605    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.256197 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255651    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.256197 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255679    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.256197 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.255843    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.256355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.256236    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.258020 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.256439    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.258020 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.256802    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.258020 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.257372    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.258020 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.257503    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.258020 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.257961    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.259716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.258960    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.259716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.259402    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.259716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.259676    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.261149 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.260772    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-web-config\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.261149 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.261097    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.261149 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.261108    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.262648 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.262604    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.262741 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.262649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.264555 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.264327    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config-out\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.264809 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.264724    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.265146 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.265075    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.265251 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.265189    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.267028 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.267010    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98mfj\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-kube-api-access-98mfj\") pod \"prometheus-k8s-0\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:12.305152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:12.304924    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:13.053568 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:13.053532    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cb38fe2-b1e1-47a5-ae73-504df175336e" path="/var/lib/kubelet/pods/4cb38fe2-b1e1-47a5-ae73-504df175336e/volumes"
Sep 18 18:46:13.779789 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:13.779758    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:46:13.781492 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:46:13.781464    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22fae9a_beee_456e_b79e_9f35f2c0bd78.slice/crio-3250b1e22bd6a24b88d9248bbc1e038eda00a400748546164387f35644d6b1b5 WatchSource:0}: Error finding container 3250b1e22bd6a24b88d9248bbc1e038eda00a400748546164387f35644d6b1b5: Status 404 returned error can't find the container with id 3250b1e22bd6a24b88d9248bbc1e038eda00a400748546164387f35644d6b1b5
Sep 18 18:46:14.597702 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.597666    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9" event={"ID":"b51f082b-ac56-416a-8532-ae1c87a60e81","Type":"ContainerStarted","Data":"66de47e5b2a81e46d28eaaaed9c1678e1c862a1b37fa7bd0be0fe22e352918d6"}
Sep 18 18:46:14.598935 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.598913    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"3250b1e22bd6a24b88d9248bbc1e038eda00a400748546164387f35644d6b1b5"}
Sep 18 18:46:14.600322 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.600296    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9" event={"ID":"66b48134-5307-4c8d-a9da-001c9fb0dd32","Type":"ContainerStarted","Data":"522738b997c2984cd323959414544ccd46cc6246842074edde4c7f0660248d79"}
Sep 18 18:46:14.600505 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.600492    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:14.605282 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.605258    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9"
Sep 18 18:46:14.618589 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.618539    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9" podStartSLOduration=2.100093069 podStartE2EDuration="4.618506668s" totalImagesPullingTime="2.518413599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:14.617829611 +0000 UTC m=+104.056089828" watchObservedRunningTime="2026-09-18 18:46:14.618506668 +0000 UTC m=+104.056766870"
Sep 18 18:46:14.637719 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:14.636804    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d9bn9" podStartSLOduration=2.043255354 podStartE2EDuration="4.636785242s" totalImagesPullingTime="2.593529888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:14.63546516 +0000 UTC m=+104.073725364" watchObservedRunningTime="2026-09-18 18:46:14.636785242 +0000 UTC m=+104.075045446"
Sep 18 18:46:15.606281 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:15.606200    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac" exitCode=0
Sep 18 18:46:15.606654 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:15.606277    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac"}
Sep 18 18:46:19.633737 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:19.633694    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6"}
Sep 18 18:46:19.634300 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:19.634272    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7"}
Sep 18 18:46:22.646853 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.646819    2568 generic.go:356] "Generic (PLEG): container finished" podID="0b078765-14df-41e5-af2c-08bd9f34612e" containerID="d5daaae4b04244d479fcf050391cef195aaa6ca92181cd31b9ae77905ce1f603" exitCode=0
Sep 18 18:46:22.647306 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.646891    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp" event={"ID":"0b078765-14df-41e5-af2c-08bd9f34612e","Type":"ContainerDied","Data":"d5daaae4b04244d479fcf050391cef195aaa6ca92181cd31b9ae77905ce1f603"}
Sep 18 18:46:22.647306 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.647243    2568 scope.go:117] "RemoveContainer" containerID="d5daaae4b04244d479fcf050391cef195aaa6ca92181cd31b9ae77905ce1f603"
Sep 18 18:46:22.650434 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.650411    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170"}
Sep 18 18:46:22.650540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.650462    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a"}
Sep 18 18:46:22.650540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.650488    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360"}
Sep 18 18:46:22.650540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.650530    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerStarted","Data":"b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61"}
Sep 18 18:46:22.700546 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:22.697865    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.27135191 podStartE2EDuration="11.697840147s" totalImagesPullingTime="7.406776671s" totalInitContainerRuntime="19.711566ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:46:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:46:22.694411074 +0000 UTC m=+112.132671287" watchObservedRunningTime="2026-09-18 18:46:22.697840147 +0000 UTC m=+112.136100350"
Sep 18 18:46:23.656788 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:23.656753    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-b9nxp" event={"ID":"0b078765-14df-41e5-af2c-08bd9f34612e","Type":"ContainerStarted","Data":"39d7b1be7d2eb0524c7a95184600a9257059fd9f307a1b3a3b62ddf81fa8ba86"}
Sep 18 18:46:27.305943 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:27.305904    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:46:30.670553 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:30.670491    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:30.670947 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:30.670606    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:42.733609 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:42.733572    2568 generic.go:356] "Generic (PLEG): container finished" podID="fa841efa-e805-4e1a-85e9-db1c65514614" containerID="0332cf1d97f515fa261a853054b9fb21b1e53bd9e4c654e9c6b9f68d53c05438" exitCode=0
Sep 18 18:46:42.734125 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:42.733651    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49" event={"ID":"fa841efa-e805-4e1a-85e9-db1c65514614","Type":"ContainerDied","Data":"0332cf1d97f515fa261a853054b9fb21b1e53bd9e4c654e9c6b9f68d53c05438"}
Sep 18 18:46:42.734125 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:42.734110    2568 scope.go:117] "RemoveContainer" containerID="0332cf1d97f515fa261a853054b9fb21b1e53bd9e4c654e9c6b9f68d53c05438"
Sep 18 18:46:43.739715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:43.739676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-ppl49" event={"ID":"fa841efa-e805-4e1a-85e9-db1c65514614","Type":"ContainerStarted","Data":"de5a164a4dab4814e9a6006e2a81bb9209a6950a259780b16afa6e1c69398f5f"}
Sep 18 18:46:50.675948 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:50.675910    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:46:50.679769 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:46:50.679749    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b4745fdf5-dvks9"
Sep 18 18:47:12.306117 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:12.306081    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:12.321645 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:12.321620    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:12.869450 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:12.869422    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:30.463165 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463128    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:47:30.463598 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463416    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="prometheus" containerID="cri-o://494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" gracePeriod=600
Sep 18 18:47:30.463598 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463476    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="thanos-sidecar" containerID="cri-o://b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61" gracePeriod=600
Sep 18 18:47:30.463598 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463464    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy" containerID="cri-o://41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a" gracePeriod=600
Sep 18 18:47:30.463598 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463504    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-web" containerID="cri-o://70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360" gracePeriod=600
Sep 18 18:47:30.463598 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463537    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="config-reloader" containerID="cri-o://2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6" gracePeriod=600
Sep 18 18:47:30.463875 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.463643    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-thanos" containerID="cri-o://85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170" gracePeriod=600
Sep 18 18:47:30.930709 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.930675    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360" exitCode=0
Sep 18 18:47:30.930709 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.930701    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61" exitCode=0
Sep 18 18:47:30.930709 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.930708    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6" exitCode=0
Sep 18 18:47:30.930982 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.930743    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360"}
Sep 18 18:47:30.930982 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.930807    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61"}
Sep 18 18:47:30.930982 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:30.930838    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6"}
Sep 18 18:47:31.939098 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:31.939004    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170" exitCode=0
Sep 18 18:47:31.939098 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:31.939029    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a" exitCode=0
Sep 18 18:47:31.939506 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:31.939087    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170"}
Sep 18 18:47:31.939506 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:31.939179    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a"}
Sep 18 18:47:32.856300 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:32.856248    2568 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 18:47:32.857229 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:32.857194    2568 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 18:47:32.858166 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:32.858140    2568 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 18:47:32.858252 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:32.858181    2568 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="prometheus" probeResult="unknown"
Sep 18 18:47:37.856424 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:37.856374    2568 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 18:47:37.857427 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:37.857391    2568 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 18:47:37.858326 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:37.858301    2568 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 18 18:47:37.858431 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:37.858336    2568 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="prometheus" probeResult="unknown"
Sep 18 18:47:38.802915 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.802888    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:38.921148 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921066    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-grpc-tls\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921148 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921125    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-0\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921160    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config-out\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921193    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-web-config\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921221    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-metrics-client-ca\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921250    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-thanos-prometheus-http-client-file\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921288    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-metrics-client-certs\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921313    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-tls\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921352    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-kube-rbac-proxy\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921377    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-2\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921415    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921446    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921471    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-serving-certs-ca-bundle\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921500    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-1\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921571    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98mfj\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-kube-api-access-98mfj\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921611    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921639    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-tls-assets\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.921682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921683    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-kubelet-serving-ca-bundle\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.922539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921710    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-db\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.922539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.921749    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-trusted-ca-bundle\") pod \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\" (UID: \"c22fae9a-beee-456e-b79e-9f35f2c0bd78\") "
Sep 18 18:47:38.922539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.922050    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.922539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.922403    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.922539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.922438    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.922801 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.922735    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.924404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.923122    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.924404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.923640    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.924404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.924155    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:47:38.926260 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.926221    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.926316 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.926285    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config-out" (OuterVolumeSpecName: "config-out") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:47:38.926391 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.926373    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:47:38.926458 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.926437    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.926538 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.926472    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-kube-api-access-98mfj" (OuterVolumeSpecName: "kube-api-access-98mfj") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "kube-api-access-98mfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:47:38.927426 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.927386    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.927552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.927447    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.927552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.927474    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.927688 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.927619    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config" (OuterVolumeSpecName: "config") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.928384 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.928352    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.928678 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.928655    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:47:38.928870 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.928852    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.938807 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.938776    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-web-config" (OuterVolumeSpecName: "web-config") pod "c22fae9a-beee-456e-b79e-9f35f2c0bd78" (UID: "c22fae9a-beee-456e-b79e-9f35f2c0bd78"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:47:38.973213 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.973183    2568 generic.go:356] "Generic (PLEG): container finished" podID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7" exitCode=0
Sep 18 18:47:38.973371 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.973224    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7"}
Sep 18 18:47:38.973371 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.973265    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:38.973371 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.973300    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c22fae9a-beee-456e-b79e-9f35f2c0bd78","Type":"ContainerDied","Data":"3250b1e22bd6a24b88d9248bbc1e038eda00a400748546164387f35644d6b1b5"}
Sep 18 18:47:38.973371 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.973351    2568 scope.go:117] "RemoveContainer" containerID="85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170"
Sep 18 18:47:38.983984 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.983965    2568 scope.go:117] "RemoveContainer" containerID="41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a"
Sep 18 18:47:38.991273 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.991253    2568 scope.go:117] "RemoveContainer" containerID="70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360"
Sep 18 18:47:38.997866 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:38.997846    2568 scope.go:117] "RemoveContainer" containerID="b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61"
Sep 18 18:47:39.004922 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.001689    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:47:39.008766 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.008740    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:47:39.010019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.009986    2568 scope.go:117] "RemoveContainer" containerID="2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6"
Sep 18 18:47:39.017045 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.017029    2568 scope.go:117] "RemoveContainer" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7"
Sep 18 18:47:39.022767 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022744    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022767 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022767    2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-grpc-tls\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022777    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022787    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config-out\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022814    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-web-config\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022823    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-metrics-client-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022831    2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022840    2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-metrics-client-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022848    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022856    2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-kube-rbac-proxy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022864    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022873    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.022881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022882    2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-config\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022890    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022899    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022907    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98mfj\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-kube-api-access-98mfj\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022915    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c22fae9a-beee-456e-b79e-9f35f2c0bd78-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022924    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c22fae9a-beee-456e-b79e-9f35f2c0bd78-tls-assets\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022932    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c22fae9a-beee-456e-b79e-9f35f2c0bd78-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.023233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.022939    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c22fae9a-beee-456e-b79e-9f35f2c0bd78-prometheus-k8s-db\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:47:39.024220 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.024210    2568 scope.go:117] "RemoveContainer" containerID="f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac"
Sep 18 18:47:39.030769 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.030751    2568 scope.go:117] "RemoveContainer" containerID="85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170"
Sep 18 18:47:39.031022 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.031002    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170\": container with ID starting with 85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170 not found: ID does not exist" containerID="85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170"
Sep 18 18:47:39.031123 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.031031    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170"} err="failed to get container status \"85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170\": rpc error: code = NotFound desc = could not find container \"85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170\": container with ID starting with 85231f99f5cbb8efa3854779e68f5bbbafb3b240dea19659846d3c68e9734170 not found: ID does not exist"
Sep 18 18:47:39.031194 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.031125    2568 scope.go:117] "RemoveContainer" containerID="41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a"
Sep 18 18:47:39.031446 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.031423    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a\": container with ID starting with 41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a not found: ID does not exist" containerID="41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a"
Sep 18 18:47:39.031564 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.031450    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a"} err="failed to get container status \"41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a\": rpc error: code = NotFound desc = could not find container \"41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a\": container with ID starting with 41aca7100cc9b2b3053232913b979bc5b67979b04eef655d991717fd5020922a not found: ID does not exist"
Sep 18 18:47:39.031564 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.031472    2568 scope.go:117] "RemoveContainer" containerID="70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360"
Sep 18 18:47:39.031865 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.031841    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360\": container with ID starting with 70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360 not found: ID does not exist" containerID="70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360"
Sep 18 18:47:39.031909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.031875    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360"} err="failed to get container status \"70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360\": rpc error: code = NotFound desc = could not find container \"70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360\": container with ID starting with 70f4d4c466ea3c33e9aee11de543a663a8caa2cab54be48b20a4bfbff5930360 not found: ID does not exist"
Sep 18 18:47:39.031909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.031893    2568 scope.go:117] "RemoveContainer" containerID="b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61"
Sep 18 18:47:39.032151 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.032134    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61\": container with ID starting with b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61 not found: ID does not exist" containerID="b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61"
Sep 18 18:47:39.032224 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032157    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61"} err="failed to get container status \"b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61\": rpc error: code = NotFound desc = could not find container \"b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61\": container with ID starting with b7db03783bf16094aa44f6734e7d5772c70af5b6b8c64bb7e2bf5bcb1c85cb61 not found: ID does not exist"
Sep 18 18:47:39.032224 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032171    2568 scope.go:117] "RemoveContainer" containerID="2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6"
Sep 18 18:47:39.032412 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.032395    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6\": container with ID starting with 2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6 not found: ID does not exist" containerID="2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6"
Sep 18 18:47:39.032467 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032421    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6"} err="failed to get container status \"2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6\": rpc error: code = NotFound desc = could not find container \"2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6\": container with ID starting with 2f12c493c11e275b058b230a3d41af02d08049cc16f1c488939ce50ccd7a45d6 not found: ID does not exist"
Sep 18 18:47:39.032467 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032445    2568 scope.go:117] "RemoveContainer" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7"
Sep 18 18:47:39.032700 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.032684    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7\": container with ID starting with 494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7 not found: ID does not exist" containerID="494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7"
Sep 18 18:47:39.032760 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032706    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7"} err="failed to get container status \"494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7\": rpc error: code = NotFound desc = could not find container \"494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7\": container with ID starting with 494efe0ab5b6682cc82780e50201a9959a63d2847ad95b40c66846878e60f0c7 not found: ID does not exist"
Sep 18 18:47:39.032760 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032724    2568 scope.go:117] "RemoveContainer" containerID="f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac"
Sep 18 18:47:39.032982 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:47:39.032965    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac\": container with ID starting with f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac not found: ID does not exist" containerID="f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac"
Sep 18 18:47:39.033018 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.032987    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac"} err="failed to get container status \"f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac\": rpc error: code = NotFound desc = could not find container \"f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac\": container with ID starting with f26709818210e9f08dc0610145f9b219a8f3980b11932e711eb6fbb7acf0a4ac not found: ID does not exist"
Sep 18 18:47:39.042306 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042286    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:47:39.042623 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042610    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="thanos-sidecar"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042624    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="thanos-sidecar"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042634    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-web"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042640    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-web"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042645    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042650    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042658    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="prometheus"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042663    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="prometheus"
Sep 18 18:47:39.042669 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042669    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="config-reloader"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042674    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="config-reloader"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042683    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="init-config-reloader"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042688    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="init-config-reloader"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042695    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-thanos"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042700    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-thanos"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042745    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-web"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042754    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="prometheus"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042761    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042767    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="kube-rbac-proxy-thanos"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042772    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="thanos-sidecar"
Sep 18 18:47:39.042895 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.042779    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" containerName="config-reloader"
Sep 18 18:47:39.046488 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.046470    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.049768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.049750    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 18:47:39.050209 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.050172    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 18:47:39.051329 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051307    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 18:47:39.051436 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051331    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 18:47:39.051436 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051329    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 18:47:39.051436 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051393    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vq29l\""
Sep 18 18:47:39.052019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051777    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 18:47:39.052019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051820    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 18:47:39.052019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051839    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 18:47:39.052019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051875    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 18:47:39.052019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.051821    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9n82qkq51vtts\""
Sep 18 18:47:39.052322 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.052229    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 18:47:39.052377 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.052322    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 18:47:39.052427 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.052327    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 18:47:39.053615 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.053597    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 18:47:39.054606 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.054586    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c22fae9a-beee-456e-b79e-9f35f2c0bd78" path="/var/lib/kubelet/pods/c22fae9a-beee-456e-b79e-9f35f2c0bd78/volumes"
Sep 18 18:47:39.058850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.058829    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 18:47:39.068196 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.068173    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:47:39.224416 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224383    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224616 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224436    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224616 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224456    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224616 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d74a03b3-7907-4975-a792-c14f534599a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224747 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224747 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224661    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224747 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224679    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224747 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224697    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224747 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224715    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224900 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224769    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224900 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224791    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224900 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224806    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqfzc\" (UniqueName: \"kubernetes.io/projected/d74a03b3-7907-4975-a792-c14f534599a9-kube-api-access-pqfzc\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224900 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224824    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.224900 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224858    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.225070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224894    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.225070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224920    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.225070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224962    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-config\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.225070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.225070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.224996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d74a03b3-7907-4975-a792-c14f534599a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.225070 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.225013    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.325842 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-config\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.325842 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325843    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325861    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d74a03b3-7907-4975-a792-c14f534599a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325896    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325921    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.325975    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d74a03b3-7907-4975-a792-c14f534599a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326067 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326541 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326496    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326644 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326586    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326644 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326784 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326784 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326784 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326742    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326784 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326745    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.326784 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326778    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.327032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326806    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqfzc\" (UniqueName: \"kubernetes.io/projected/d74a03b3-7907-4975-a792-c14f534599a9-kube-api-access-pqfzc\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.327032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326836    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.327032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326881    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.327032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326906    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.327032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.326929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.327294 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.327252    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329449 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.328667    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-config\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329449 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.328849    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d74a03b3-7907-4975-a792-c14f534599a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329449 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.328973    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d74a03b3-7907-4975-a792-c14f534599a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329449 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.329056    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329449 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.329265    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329449 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.329366    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329882 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.329471    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.329882 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.329782    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.330015 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.329973    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.330235 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.330207    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.330481 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.330456    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.330617 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.330598    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.331442 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.331421    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.331658 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.331642    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.332302 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.332284    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d74a03b3-7907-4975-a792-c14f534599a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.332832 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.332814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d74a03b3-7907-4975-a792-c14f534599a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.338826 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.338806    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqfzc\" (UniqueName: \"kubernetes.io/projected/d74a03b3-7907-4975-a792-c14f534599a9-kube-api-access-pqfzc\") pod \"prometheus-k8s-0\" (UID: \"d74a03b3-7907-4975-a792-c14f534599a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.357763 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.357738    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:47:39.641417 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.641390    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 18:47:39.643023 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:47:39.642993    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74a03b3_7907_4975_a792_c14f534599a9.slice/crio-b2c6f4e4c0f1843a5d46427b71b7ea7e21aad543c05c2f654bf2dee5716e3254 WatchSource:0}: Error finding container b2c6f4e4c0f1843a5d46427b71b7ea7e21aad543c05c2f654bf2dee5716e3254: Status 404 returned error can't find the container with id b2c6f4e4c0f1843a5d46427b71b7ea7e21aad543c05c2f654bf2dee5716e3254
Sep 18 18:47:39.979404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.979373    2568 generic.go:356] "Generic (PLEG): container finished" podID="d74a03b3-7907-4975-a792-c14f534599a9" containerID="9dc7f545e59d83ade8c4eade40984e8f9d9e33cc8cbac6005ae9f10c8f07774e" exitCode=0
Sep 18 18:47:39.979876 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.979459    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerDied","Data":"9dc7f545e59d83ade8c4eade40984e8f9d9e33cc8cbac6005ae9f10c8f07774e"}
Sep 18 18:47:39.979876 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:39.979567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"b2c6f4e4c0f1843a5d46427b71b7ea7e21aad543c05c2f654bf2dee5716e3254"}
Sep 18 18:47:40.988280 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:40.988237    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"081584c2defbd5f3a4cacf6ee6144538fd41f80c31747a2dfeae08252eff49b4"}
Sep 18 18:47:40.988715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:40.988325    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"bdf5af14438b3216aea455a548f03e62f7c6499d6a57a87d3175994f2a40db87"}
Sep 18 18:47:40.988715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:40.988365    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"ed82e3f677733ccd5277383b9f348900897bfce8ce4406cac3846aa97d79d7ee"}
Sep 18 18:47:40.988715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:40.988403    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"e54706d472f3e97e447462ecd1735ad4b0c3b89616bf4f27790f05fe685ccd5e"}
Sep 18 18:47:40.988715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:40.988440    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"691ac7257131d2ba90a3f11523f1b789d6be9ee7b31ebc5b922556e1870f6304"}
Sep 18 18:47:40.988715 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:40.988477    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d74a03b3-7907-4975-a792-c14f534599a9","Type":"ContainerStarted","Data":"0d0935f27f76e9c764503c57c690ae473ad1999fca32bef7ba1df5d16a583936"}
Sep 18 18:47:41.030099 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:41.026773    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.008618548 podStartE2EDuration="2.026748359s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.129811ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:47:41.023743253 +0000 UTC m=+190.462003468" watchObservedRunningTime="2026-09-18 18:47:41.026748359 +0000 UTC m=+190.465008564"
Sep 18 18:47:44.358200 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:47:44.358165    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:48:39.358819 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:48:39.358789    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:48:39.374198 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:48:39.374169    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:48:40.240027 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:48:40.240000    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 18:49:30.935404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:49:30.935373    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:49:30.935893 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:49:30.935455    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:49:30.946281 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:49:30.946262    2568 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 18:50:50.506162 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.506123    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"]
Sep 18 18:50:50.508493 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.508464    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.515271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.515249    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 18:50:50.516171 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.516152    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 18:50:50.516271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.516179    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 18:50:50.516271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.516231    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 18:50:50.516271 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.516179    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:50:50.516860 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.516837    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cv7vk\""
Sep 18 18:50:50.523152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.523132    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"]
Sep 18 18:50:50.649765 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.649724    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/997a9b51-8512-4a2f-b15d-6193dbd3ece6-cert\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.649951 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.649779    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/997a9b51-8512-4a2f-b15d-6193dbd3ece6-metrics-cert\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.649951 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.649839    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wfxc\" (UniqueName: \"kubernetes.io/projected/997a9b51-8512-4a2f-b15d-6193dbd3ece6-kube-api-access-8wfxc\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.649951 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.649939    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/997a9b51-8512-4a2f-b15d-6193dbd3ece6-manager-config\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.751123 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.751081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/997a9b51-8512-4a2f-b15d-6193dbd3ece6-manager-config\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.751333 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.751148    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/997a9b51-8512-4a2f-b15d-6193dbd3ece6-cert\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.751333 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.751178    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/997a9b51-8512-4a2f-b15d-6193dbd3ece6-metrics-cert\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.751333 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.751194    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wfxc\" (UniqueName: \"kubernetes.io/projected/997a9b51-8512-4a2f-b15d-6193dbd3ece6-kube-api-access-8wfxc\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.751867 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.751845    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/997a9b51-8512-4a2f-b15d-6193dbd3ece6-manager-config\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.753585 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.753555    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/997a9b51-8512-4a2f-b15d-6193dbd3ece6-cert\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.753670 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.753616    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/997a9b51-8512-4a2f-b15d-6193dbd3ece6-metrics-cert\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.760205 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.760153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wfxc\" (UniqueName: \"kubernetes.io/projected/997a9b51-8512-4a2f-b15d-6193dbd3ece6-kube-api-access-8wfxc\") pod \"lws-controller-manager-bf466d58c-ghhdd\" (UID: \"997a9b51-8512-4a2f-b15d-6193dbd3ece6\") " pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:50.817917 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:50.817883    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:51.086176 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:51.086151    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"]
Sep 18 18:50:51.089977 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:50:51.089585    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997a9b51_8512_4a2f_b15d_6193dbd3ece6.slice/crio-e1251b19caa6028fe42458d5d992cb5e41e90775d974f84ca6a292b807a70c94 WatchSource:0}: Error finding container e1251b19caa6028fe42458d5d992cb5e41e90775d974f84ca6a292b807a70c94: Status 404 returned error can't find the container with id e1251b19caa6028fe42458d5d992cb5e41e90775d974f84ca6a292b807a70c94
Sep 18 18:50:51.095880 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:51.092668    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:50:51.736692 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:51.736656    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd" event={"ID":"997a9b51-8512-4a2f-b15d-6193dbd3ece6","Type":"ContainerStarted","Data":"e1251b19caa6028fe42458d5d992cb5e41e90775d974f84ca6a292b807a70c94"}
Sep 18 18:50:58.765540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:58.765496    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd" event={"ID":"997a9b51-8512-4a2f-b15d-6193dbd3ece6","Type":"ContainerStarted","Data":"3fa697bb71b8b3f34c7ccf279e709faa8cc64e9222a2261fbb4c578313b96c63"}
Sep 18 18:50:58.765918 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:58.765591    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:50:58.789190 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:50:58.789133    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd" podStartSLOduration=1.6375724200000001 podStartE2EDuration="8.789117589s" totalImagesPullingTime="7.151545169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:50:58.788095947 +0000 UTC m=+388.226356151" watchObservedRunningTime="2026-09-18 18:50:58.789117589 +0000 UTC m=+388.227377791"
Sep 18 18:51:02.742600 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.742556    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"]
Sep 18 18:51:02.745013 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.744998    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.751101 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.751061    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 18:51:02.751247 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.751165    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 18:51:02.751606 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.751276    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 18:51:02.753178 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.753161    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 18:51:02.753275 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.753213    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t6jkl\""
Sep 18 18:51:02.860065 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.860023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2124a540-3613-467a-8373-4b124a547095-tmp\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.860255 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.860090    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2124a540-3613-467a-8373-4b124a547095-manifests\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.860255 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.860125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2124a540-3613-467a-8373-4b124a547095-webhook-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.860255 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.860160    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2124a540-3613-467a-8373-4b124a547095-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.860255 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.860199    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx2w4\" (UniqueName: \"kubernetes.io/projected/2124a540-3613-467a-8373-4b124a547095-kube-api-access-gx2w4\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.870117 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.870084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"]
Sep 18 18:51:02.961066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.961026    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2124a540-3613-467a-8373-4b124a547095-tmp\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.961267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.961087    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2124a540-3613-467a-8373-4b124a547095-manifests\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.961267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.961112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2124a540-3613-467a-8373-4b124a547095-webhook-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.961267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.961140    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2124a540-3613-467a-8373-4b124a547095-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.961267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.961162    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx2w4\" (UniqueName: \"kubernetes.io/projected/2124a540-3613-467a-8373-4b124a547095-kube-api-access-gx2w4\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.961552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.961503    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2124a540-3613-467a-8373-4b124a547095-manifests\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.963243 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.963222    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2124a540-3613-467a-8373-4b124a547095-tmp\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.963461 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.963445    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2124a540-3613-467a-8373-4b124a547095-webhook-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.963551 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.963530    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2124a540-3613-467a-8373-4b124a547095-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:02.974115 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:02.974082    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx2w4\" (UniqueName: \"kubernetes.io/projected/2124a540-3613-467a-8373-4b124a547095-kube-api-access-gx2w4\") pod \"opendatahub-operator-controller-manager-699fb7c78d-6dqvz\" (UID: \"2124a540-3613-467a-8373-4b124a547095\") " pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:03.055411 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:03.055336    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:03.350803 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:03.350726    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"]
Sep 18 18:51:03.354432 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:51:03.354397    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2124a540_3613_467a_8373_4b124a547095.slice/crio-08cee7c098f8eb0304f0ce6bfffddf240880aef9a241176d1a7479835cf63064 WatchSource:0}: Error finding container 08cee7c098f8eb0304f0ce6bfffddf240880aef9a241176d1a7479835cf63064: Status 404 returned error can't find the container with id 08cee7c098f8eb0304f0ce6bfffddf240880aef9a241176d1a7479835cf63064
Sep 18 18:51:03.793034 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:03.792993    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz" event={"ID":"2124a540-3613-467a-8373-4b124a547095","Type":"ContainerStarted","Data":"08cee7c098f8eb0304f0ce6bfffddf240880aef9a241176d1a7479835cf63064"}
Sep 18 18:51:06.812256 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:06.812211    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz" event={"ID":"2124a540-3613-467a-8373-4b124a547095","Type":"ContainerStarted","Data":"7a0d7fa00821f6763afa9766d1388235d1f31164a5e395964d1c15cd5188971f"}
Sep 18 18:51:07.818535 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:07.818483    2568 generic.go:356] "Generic (PLEG): container finished" podID="2124a540-3613-467a-8373-4b124a547095" containerID="7a0d7fa00821f6763afa9766d1388235d1f31164a5e395964d1c15cd5188971f" exitCode=0
Sep 18 18:51:07.818891 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:07.818550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz" event={"ID":"2124a540-3613-467a-8373-4b124a547095","Type":"ContainerDied","Data":"7a0d7fa00821f6763afa9766d1388235d1f31164a5e395964d1c15cd5188971f"}
Sep 18 18:51:08.825111 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:08.825076    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz" event={"ID":"2124a540-3613-467a-8373-4b124a547095","Type":"ContainerStarted","Data":"6d0327b39fa34992068705ecebcb6a2318f2dcdb665872d342c16b8f73e87c06"}
Sep 18 18:51:08.825483 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:08.825179    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:08.857783 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:08.857737    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz" podStartSLOduration=2.454072892 podStartE2EDuration="6.857719524s" totalImagesPullingTime="3.368708436s" totalInitContainerRuntime="1.034938196s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:08.855091516 +0000 UTC m=+398.293351719" watchObservedRunningTime="2026-09-18 18:51:08.857719524 +0000 UTC m=+398.295979726"
Sep 18 18:51:09.771939 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:09.771910    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bf466d58c-ghhdd"
Sep 18 18:51:19.832664 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:19.832632    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-699fb7c78d-6dqvz"
Sep 18 18:51:30.444149 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.444108    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"]
Sep 18 18:51:30.447551 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.447528    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.449921 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.449889    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-m8zz6\""
Sep 18 18:51:30.450069 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.450042    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:51:30.463721 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.463691    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"]
Sep 18 18:51:30.504432 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504400    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504432 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504439    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88efd088-574b-4128-80c9-5554600d3baf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504469    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504604    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6qz\" (UniqueName: \"kubernetes.io/projected/88efd088-574b-4128-80c9-5554600d3baf-kube-api-access-9m6qz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504625    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504653    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88efd088-574b-4128-80c9-5554600d3baf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504845 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.504845 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.504748    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88efd088-574b-4128-80c9-5554600d3baf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605430 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605387    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605430 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88efd088-574b-4128-80c9-5554600d3baf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605452    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605486    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88efd088-574b-4128-80c9-5554600d3baf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605581    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88efd088-574b-4128-80c9-5554600d3baf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605630    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605654    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.605713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6qz\" (UniqueName: \"kubernetes.io/projected/88efd088-574b-4128-80c9-5554600d3baf-kube-api-access-9m6qz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.606081 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605920    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.606081 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.605941    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.606081 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.606028    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.606259 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.606161    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.606259 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.606200    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88efd088-574b-4128-80c9-5554600d3baf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.607876 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.607851    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88efd088-574b-4128-80c9-5554600d3baf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.608105 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.608088    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88efd088-574b-4128-80c9-5554600d3baf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.613295 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.613267    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88efd088-574b-4128-80c9-5554600d3baf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.613386 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.613321    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6qz\" (UniqueName: \"kubernetes.io/projected/88efd088-574b-4128-80c9-5554600d3baf-kube-api-access-9m6qz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk\" (UID: \"88efd088-574b-4128-80c9-5554600d3baf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:30.757583 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:30.757536    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:31.034464 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:31.034430    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"]
Sep 18 18:51:31.036966 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:51:31.036938    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88efd088_574b_4128_80c9_5554600d3baf.slice/crio-e9e439b736e67f43483f6ddbbb26f7d36e8c13ffb89a6ac10aa211168578e145 WatchSource:0}: Error finding container e9e439b736e67f43483f6ddbbb26f7d36e8c13ffb89a6ac10aa211168578e145: Status 404 returned error can't find the container with id e9e439b736e67f43483f6ddbbb26f7d36e8c13ffb89a6ac10aa211168578e145
Sep 18 18:51:31.926878 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:31.926842    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk" event={"ID":"88efd088-574b-4128-80c9-5554600d3baf","Type":"ContainerStarted","Data":"e9e439b736e67f43483f6ddbbb26f7d36e8c13ffb89a6ac10aa211168578e145"}
Sep 18 18:51:33.129184 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.127717    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7mjbt"]
Sep 18 18:51:33.138557 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.138497    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.142263 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.142082    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 18:51:33.142408 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.142384    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ms4f6\""
Sep 18 18:51:33.149930 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.149897    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7mjbt"]
Sep 18 18:51:33.228756 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.228717    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-cert\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.228961 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.228787    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b4rh\" (UniqueName: \"kubernetes.io/projected/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-kube-api-access-5b4rh\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.330293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.330258    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-cert\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.330565 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.330316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b4rh\" (UniqueName: \"kubernetes.io/projected/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-kube-api-access-5b4rh\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.330565 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:51:33.330431    2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 18 18:51:33.330565 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:51:33.330550    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-cert podName:8e629ed7-eab2-4cda-b3fd-0ede8a90f89a nodeName:}" failed. No retries permitted until 2026-09-18 18:51:33.830504263 +0000 UTC m=+423.268764446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-cert") pod "odh-model-controller-7998c67dc4-7mjbt" (UID: "8e629ed7-eab2-4cda-b3fd-0ede8a90f89a") : secret "odh-model-controller-webhook-cert" not found
Sep 18 18:51:33.342206 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.342175    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b4rh\" (UniqueName: \"kubernetes.io/projected/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-kube-api-access-5b4rh\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.611571 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.611535    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:51:33.611647 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.611607    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:51:33.611647 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.611636    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:51:33.757681 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.757647    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-prjmf"]
Sep 18 18:51:33.761156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.761128    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.763537 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.763498    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 18:51:33.763660 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.763509    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-pcvrg\""
Sep 18 18:51:33.775700 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.775669    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-prjmf"]
Sep 18 18:51:33.836194 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.836144    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8vxv\" (UniqueName: \"kubernetes.io/projected/dc0f4ba6-c961-4010-ac60-4f834224caa5-kube-api-access-b8vxv\") pod \"model-serving-api-54d89bd5fc-prjmf\" (UID: \"dc0f4ba6-c961-4010-ac60-4f834224caa5\") " pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.836372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.836268    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-cert\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.836372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.836333    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f4ba6-c961-4010-ac60-4f834224caa5-tls-certs\") pod \"model-serving-api-54d89bd5fc-prjmf\" (UID: \"dc0f4ba6-c961-4010-ac60-4f834224caa5\") " pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.838657 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.838631    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e629ed7-eab2-4cda-b3fd-0ede8a90f89a-cert\") pod \"odh-model-controller-7998c67dc4-7mjbt\" (UID: \"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:33.936846 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.936753    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f4ba6-c961-4010-ac60-4f834224caa5-tls-certs\") pod \"model-serving-api-54d89bd5fc-prjmf\" (UID: \"dc0f4ba6-c961-4010-ac60-4f834224caa5\") " pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.937014 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.936876    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8vxv\" (UniqueName: \"kubernetes.io/projected/dc0f4ba6-c961-4010-ac60-4f834224caa5-kube-api-access-b8vxv\") pod \"model-serving-api-54d89bd5fc-prjmf\" (UID: \"dc0f4ba6-c961-4010-ac60-4f834224caa5\") " pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.937014 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.936953    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk" event={"ID":"88efd088-574b-4128-80c9-5554600d3baf","Type":"ContainerStarted","Data":"64969acd08c8539324de68124a0dfba9ea110d9a91d0f79319929f0d38ccf2b1"}
Sep 18 18:51:33.939173 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.939144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0f4ba6-c961-4010-ac60-4f834224caa5-tls-certs\") pod \"model-serving-api-54d89bd5fc-prjmf\" (UID: \"dc0f4ba6-c961-4010-ac60-4f834224caa5\") " pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.945355 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.945331    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8vxv\" (UniqueName: \"kubernetes.io/projected/dc0f4ba6-c961-4010-ac60-4f834224caa5-kube-api-access-b8vxv\") pod \"model-serving-api-54d89bd5fc-prjmf\" (UID: \"dc0f4ba6-c961-4010-ac60-4f834224caa5\") " pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:33.958853 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:33.958805    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk" podStartSLOduration=1.386641582 podStartE2EDuration="3.958786266s" totalImagesPullingTime="2.572144684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:33.956997732 +0000 UTC m=+423.395257945" watchObservedRunningTime="2026-09-18 18:51:33.958786266 +0000 UTC m=+423.397046469"
Sep 18 18:51:34.054801 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.054767    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:34.080066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.080030    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:34.328892 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.328868    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7mjbt"]
Sep 18 18:51:34.331484 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:51:34.331461    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e629ed7_eab2_4cda_b3fd_0ede8a90f89a.slice/crio-27435d1fd25007bfc588fa6d33acc178fb6d17969ee3583ba8b7b781ffc6dc6e WatchSource:0}: Error finding container 27435d1fd25007bfc588fa6d33acc178fb6d17969ee3583ba8b7b781ffc6dc6e: Status 404 returned error can't find the container with id 27435d1fd25007bfc588fa6d33acc178fb6d17969ee3583ba8b7b781ffc6dc6e
Sep 18 18:51:34.365114 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.365086    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-prjmf"]
Sep 18 18:51:34.368118 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:51:34.368078    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0f4ba6_c961_4010_ac60_4f834224caa5.slice/crio-f8aff99a73df4702da2cd4d665b106c41a6f95f5545fcb2ffbd906beaa5b3cfe WatchSource:0}: Error finding container f8aff99a73df4702da2cd4d665b106c41a6f95f5545fcb2ffbd906beaa5b3cfe: Status 404 returned error can't find the container with id f8aff99a73df4702da2cd4d665b106c41a6f95f5545fcb2ffbd906beaa5b3cfe
Sep 18 18:51:34.758374 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.758338    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:34.762920 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.762895    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:34.943014 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.942978    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt" event={"ID":"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a","Type":"ContainerStarted","Data":"27435d1fd25007bfc588fa6d33acc178fb6d17969ee3583ba8b7b781ffc6dc6e"}
Sep 18 18:51:34.944190 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.944166    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf" event={"ID":"dc0f4ba6-c961-4010-ac60-4f834224caa5","Type":"ContainerStarted","Data":"f8aff99a73df4702da2cd4d665b106c41a6f95f5545fcb2ffbd906beaa5b3cfe"}
Sep 18 18:51:34.944362 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.944344    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:34.945315 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:34.945297    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk"
Sep 18 18:51:38.966565 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:38.966506    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt" event={"ID":"8e629ed7-eab2-4cda-b3fd-0ede8a90f89a","Type":"ContainerStarted","Data":"3a242dc2ba6146858b3f510591ab21bb016c4ee9b3885e1e3f5928e0aa1449bf"}
Sep 18 18:51:38.967266 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:38.966594    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:38.968111 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:38.968091    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf" event={"ID":"dc0f4ba6-c961-4010-ac60-4f834224caa5","Type":"ContainerStarted","Data":"a7cd1dd0a847a5067e7095ef940436421143d3f208ee56e6943c28b33ff009c6"}
Sep 18 18:51:38.968237 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:38.968220    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:38.993785 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:38.993723    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt" podStartSLOduration=2.221828105 podStartE2EDuration="5.993701057s" totalImagesPullingTime="3.771872952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:38.989182082 +0000 UTC m=+428.427442284" watchObservedRunningTime="2026-09-18 18:51:38.993701057 +0000 UTC m=+428.431961262"
Sep 18 18:51:39.029348 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:39.029294    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf" podStartSLOduration=2.33838846 podStartE2EDuration="6.029276775s" totalImagesPullingTime="3.690888315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:39.027772028 +0000 UTC m=+428.466032254" watchObservedRunningTime="2026-09-18 18:51:39.029276775 +0000 UTC m=+428.467536978"
Sep 18 18:51:40.761942 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.761909    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"]
Sep 18 18:51:40.765351 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.765334    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.768674 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.768647    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 18:51:40.768674 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.768648    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 18:51:40.768985 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.768969    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 18:51:40.769030 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.768982    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-knzqd\""
Sep 18 18:51:40.797373 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.797344    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"]
Sep 18 18:51:40.902962 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.902928    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.903135 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.902969    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wswll\" (UniqueName: \"kubernetes.io/projected/315991ff-9db3-466e-bb57-5708a6d66a33-kube-api-access-wswll\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.903135 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.903032    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.903135 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.903068    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/315991ff-9db3-466e-bb57-5708a6d66a33-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.903267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.903137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.903267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.903169    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/315991ff-9db3-466e-bb57-5708a6d66a33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:40.903267 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:40.903201    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/315991ff-9db3-466e-bb57-5708a6d66a33-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004048 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004013    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004048 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wswll\" (UniqueName: \"kubernetes.io/projected/315991ff-9db3-466e-bb57-5708a6d66a33-kube-api-access-wswll\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004077    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/315991ff-9db3-466e-bb57-5708a6d66a33-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004184    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/315991ff-9db3-466e-bb57-5708a6d66a33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.004288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004221    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/315991ff-9db3-466e-bb57-5708a6d66a33-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.005032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.004995    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/315991ff-9db3-466e-bb57-5708a6d66a33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.006528 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.006495    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/315991ff-9db3-466e-bb57-5708a6d66a33-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.006991 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.006968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.007092 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.007013    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.007092 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.007034    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/315991ff-9db3-466e-bb57-5708a6d66a33-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.014379 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.014323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/315991ff-9db3-466e-bb57-5708a6d66a33-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.014379 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.014323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wswll\" (UniqueName: \"kubernetes.io/projected/315991ff-9db3-466e-bb57-5708a6d66a33-kube-api-access-wswll\") pod \"istiod-openshift-gateway-5677ccb49b-tqd44\" (UID: \"315991ff-9db3-466e-bb57-5708a6d66a33\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.074161 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.074135    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:41.377015 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.376991    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"]
Sep 18 18:51:41.380440 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:51:41.379931    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315991ff_9db3_466e_bb57_5708a6d66a33.slice/crio-d11785d4191846d5919fe32f4c64972116319182cb5a47bf6d38fca3b2464313 WatchSource:0}: Error finding container d11785d4191846d5919fe32f4c64972116319182cb5a47bf6d38fca3b2464313: Status 404 returned error can't find the container with id d11785d4191846d5919fe32f4c64972116319182cb5a47bf6d38fca3b2464313
Sep 18 18:51:41.983246 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:41.983214    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44" event={"ID":"315991ff-9db3-466e-bb57-5708a6d66a33","Type":"ContainerStarted","Data":"d11785d4191846d5919fe32f4c64972116319182cb5a47bf6d38fca3b2464313"}
Sep 18 18:51:48.313877 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:48.313828    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:51:48.314299 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:48.313897    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:51:49.017041 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.016998    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44" event={"ID":"315991ff-9db3-466e-bb57-5708a6d66a33","Type":"ContainerStarted","Data":"ef89954d2629b00649d9cfa2eb004ebb8592397ff01f3ccb26d10a7bb22abf87"}
Sep 18 18:51:49.017227 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.017155    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:51:49.018884 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.018860    2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-tqd44 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:51:49.019002 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.018906    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44" podUID="315991ff-9db3-466e-bb57-5708a6d66a33" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:51:49.043318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.043262    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44" podStartSLOduration=2.111884268 podStartE2EDuration="9.043247388s" totalImagesPullingTime="6.93136312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:51:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:51:49.042870243 +0000 UTC m=+438.481130462" watchObservedRunningTime="2026-09-18 18:51:49.043247388 +0000 UTC m=+438.481507592"
Sep 18 18:51:49.975591 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.975549    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7mjbt"
Sep 18 18:51:49.977477 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:49.977455    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-prjmf"
Sep 18 18:51:50.023184 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:51:50.023151    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-tqd44"
Sep 18 18:54:30.972081 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:30.972051    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:54:30.972606 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:30.972306    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:54:40.828345 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.828301    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77ccc6d5bb-q75jb"]
Sep 18 18:54:40.831872 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.831848    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:40.834132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.834086    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 18:54:40.834132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.834088    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 18:54:40.834132 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.834123    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:54:40.834441 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.834360    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pcg7z\""
Sep 18 18:54:40.841713 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.841687    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77ccc6d5bb-q75jb"]
Sep 18 18:54:40.938823 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.938787    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-metrics-certs\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:40.938997 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.938853    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fvs4\" (UniqueName: \"kubernetes.io/projected/3186d5da-db2a-4304-8c58-27adf7785114-kube-api-access-8fvs4\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:40.938997 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:40.938912    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.039532 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.039478    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-metrics-certs\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.039706 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.039573    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fvs4\" (UniqueName: \"kubernetes.io/projected/3186d5da-db2a-4304-8c58-27adf7785114-kube-api-access-8fvs4\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.039706 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.039610    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.039778 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:54:41.039743    2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 18 18:54:41.039812 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:54:41.039808    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert podName:3186d5da-db2a-4304-8c58-27adf7785114 nodeName:}" failed. No retries permitted until 2026-09-18 18:54:41.539789043 +0000 UTC m=+610.978049225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert") pod "maas-controller-77ccc6d5bb-q75jb" (UID: "3186d5da-db2a-4304-8c58-27adf7785114") : secret "maas-controller-webhook-cert" not found
Sep 18 18:54:41.041985 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.041958    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-metrics-certs\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.060889 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.060862    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fvs4\" (UniqueName: \"kubernetes.io/projected/3186d5da-db2a-4304-8c58-27adf7785114-kube-api-access-8fvs4\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.543338 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.543298    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.545805 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.545781    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert\") pod \"maas-controller-77ccc6d5bb-q75jb\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") " pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:41.742303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:41.742255    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:42.014970 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:42.014947    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77ccc6d5bb-q75jb"]
Sep 18 18:54:42.016786 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:54:42.016764    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3186d5da_db2a_4304_8c58_27adf7785114.slice/crio-6bff02657b5fdb97c3f9bf3eea793ccbad57fa8594b4fab23ffa894e660b7ed1 WatchSource:0}: Error finding container 6bff02657b5fdb97c3f9bf3eea793ccbad57fa8594b4fab23ffa894e660b7ed1: Status 404 returned error can't find the container with id 6bff02657b5fdb97c3f9bf3eea793ccbad57fa8594b4fab23ffa894e660b7ed1
Sep 18 18:54:42.803152 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:42.803116    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb" event={"ID":"3186d5da-db2a-4304-8c58-27adf7785114","Type":"ContainerStarted","Data":"6bff02657b5fdb97c3f9bf3eea793ccbad57fa8594b4fab23ffa894e660b7ed1"}
Sep 18 18:54:44.814459 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:44.814368    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb" event={"ID":"3186d5da-db2a-4304-8c58-27adf7785114","Type":"ContainerStarted","Data":"b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48"}
Sep 18 18:54:44.814864 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:44.814489    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:44.834453 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:44.834396    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb" podStartSLOduration=2.295635734 podStartE2EDuration="4.834377465s" totalImagesPullingTime="2.538741731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:54:44.833429863 +0000 UTC m=+614.271690081" watchObservedRunningTime="2026-09-18 18:54:44.834377465 +0000 UTC m=+614.272637671"
Sep 18 18:54:55.121666 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.121590    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77ccc6d5bb-q75jb"]
Sep 18 18:54:55.122202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.121788    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb" podUID="3186d5da-db2a-4304-8c58-27adf7785114" containerName="manager" containerID="cri-o://b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48" gracePeriod=30
Sep 18 18:54:55.123455 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.123432    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:55.369881 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.369854    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:55.460211 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.460129    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fvs4\" (UniqueName: \"kubernetes.io/projected/3186d5da-db2a-4304-8c58-27adf7785114-kube-api-access-8fvs4\") pod \"3186d5da-db2a-4304-8c58-27adf7785114\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") "
Sep 18 18:54:55.460211 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.460193    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-metrics-certs\") pod \"3186d5da-db2a-4304-8c58-27adf7785114\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") "
Sep 18 18:54:55.460426 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.460217    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert\") pod \"3186d5da-db2a-4304-8c58-27adf7785114\" (UID: \"3186d5da-db2a-4304-8c58-27adf7785114\") "
Sep 18 18:54:55.462362 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.462324    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3186d5da-db2a-4304-8c58-27adf7785114-kube-api-access-8fvs4" (OuterVolumeSpecName: "kube-api-access-8fvs4") pod "3186d5da-db2a-4304-8c58-27adf7785114" (UID: "3186d5da-db2a-4304-8c58-27adf7785114"). InnerVolumeSpecName "kube-api-access-8fvs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:54:55.462494 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.462358    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3186d5da-db2a-4304-8c58-27adf7785114" (UID: "3186d5da-db2a-4304-8c58-27adf7785114"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:54:55.462494 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.462392    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3186d5da-db2a-4304-8c58-27adf7785114" (UID: "3186d5da-db2a-4304-8c58-27adf7785114"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:54:55.561061 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.561015    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-metrics-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:54:55.561061 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.561054    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3186d5da-db2a-4304-8c58-27adf7785114-webhook-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:54:55.561061 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.561064    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fvs4\" (UniqueName: \"kubernetes.io/projected/3186d5da-db2a-4304-8c58-27adf7785114-kube-api-access-8fvs4\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:54:55.866623 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.866584    2568 generic.go:356] "Generic (PLEG): container finished" podID="3186d5da-db2a-4304-8c58-27adf7785114" containerID="b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48" exitCode=0
Sep 18 18:54:55.866789 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.866633    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb"
Sep 18 18:54:55.866789 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.866670    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb" event={"ID":"3186d5da-db2a-4304-8c58-27adf7785114","Type":"ContainerDied","Data":"b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48"}
Sep 18 18:54:55.866789 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.866721    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77ccc6d5bb-q75jb" event={"ID":"3186d5da-db2a-4304-8c58-27adf7785114","Type":"ContainerDied","Data":"6bff02657b5fdb97c3f9bf3eea793ccbad57fa8594b4fab23ffa894e660b7ed1"}
Sep 18 18:54:55.866789 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.866744    2568 scope.go:117] "RemoveContainer" containerID="b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48"
Sep 18 18:54:55.875813 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.875796    2568 scope.go:117] "RemoveContainer" containerID="b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48"
Sep 18 18:54:55.876083 ip-10-0-132-94 kubenswrapper[2568]: E0918 18:54:55.876066    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48\": container with ID starting with b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48 not found: ID does not exist" containerID="b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48"
Sep 18 18:54:55.876136 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.876092    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48"} err="failed to get container status \"b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48\": rpc error: code = NotFound desc = could not find container \"b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48\": container with ID starting with b0cf596f04e7e754950d363b9434d394fe3c3cf5e34bd52ce24683529f62fe48 not found: ID does not exist"
Sep 18 18:54:55.888038 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.888012    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77ccc6d5bb-q75jb"]
Sep 18 18:54:55.891651 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:55.891624    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77ccc6d5bb-q75jb"]
Sep 18 18:54:57.054241 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:54:57.054201    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3186d5da-db2a-4304-8c58-27adf7785114" path="/var/lib/kubelet/pods/3186d5da-db2a-4304-8c58-27adf7785114/volumes"
Sep 18 18:55:17.801468 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.801426    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"]
Sep 18 18:55:17.804072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.801958    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3186d5da-db2a-4304-8c58-27adf7785114" containerName="manager"
Sep 18 18:55:17.804072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.801974    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3186d5da-db2a-4304-8c58-27adf7785114" containerName="manager"
Sep 18 18:55:17.804072 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.802035    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3186d5da-db2a-4304-8c58-27adf7785114" containerName="manager"
Sep 18 18:55:17.804949 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.804920    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:17.807293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.807267    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r8zzg\""
Sep 18 18:55:17.807397 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.807288    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 18:55:17.815722 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.815700    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"]
Sep 18 18:55:17.863726 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.863695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/268b3d97-09f9-458d-9204-d460a6112881-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kjc76\" (UID: \"268b3d97-09f9-458d-9204-d460a6112881\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:17.863885 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.863752    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdkn\" (UniqueName: \"kubernetes.io/projected/268b3d97-09f9-458d-9204-d460a6112881-kube-api-access-2cdkn\") pod \"payload-pre-processing-59fbc47bf8-kjc76\" (UID: \"268b3d97-09f9-458d-9204-d460a6112881\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:17.964540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.964482    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/268b3d97-09f9-458d-9204-d460a6112881-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kjc76\" (UID: \"268b3d97-09f9-458d-9204-d460a6112881\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:17.964710 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.964609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdkn\" (UniqueName: \"kubernetes.io/projected/268b3d97-09f9-458d-9204-d460a6112881-kube-api-access-2cdkn\") pod \"payload-pre-processing-59fbc47bf8-kjc76\" (UID: \"268b3d97-09f9-458d-9204-d460a6112881\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:17.966138 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.966112    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/268b3d97-09f9-458d-9204-d460a6112881-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kjc76\" (UID: \"268b3d97-09f9-458d-9204-d460a6112881\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:17.973091 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:17.973067    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdkn\" (UniqueName: \"kubernetes.io/projected/268b3d97-09f9-458d-9204-d460a6112881-kube-api-access-2cdkn\") pod \"payload-pre-processing-59fbc47bf8-kjc76\" (UID: \"268b3d97-09f9-458d-9204-d460a6112881\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:18.116954 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:18.116864    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:18.393050 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:18.393021    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"]
Sep 18 18:55:18.395915 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:55:18.395886    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod268b3d97_09f9_458d_9204_d460a6112881.slice/crio-ac52bd1e5d8af8aa489cd7432a6b516cb52947cbf7360709035c77f90400bdd8 WatchSource:0}: Error finding container ac52bd1e5d8af8aa489cd7432a6b516cb52947cbf7360709035c77f90400bdd8: Status 404 returned error can't find the container with id ac52bd1e5d8af8aa489cd7432a6b516cb52947cbf7360709035c77f90400bdd8
Sep 18 18:55:18.976533 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:18.976478    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76" event={"ID":"268b3d97-09f9-458d-9204-d460a6112881","Type":"ContainerStarted","Data":"ac52bd1e5d8af8aa489cd7432a6b516cb52947cbf7360709035c77f90400bdd8"}
Sep 18 18:55:20.987736 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:20.987699    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76" event={"ID":"268b3d97-09f9-458d-9204-d460a6112881","Type":"ContainerStarted","Data":"5601799b134e202ae9c972817b91a596811342be2f89ccc316dd81e9e8019295"}
Sep 18 18:55:20.988124 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:20.987843    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:21.004365 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:21.004317    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76" podStartSLOduration=1.696439464 podStartE2EDuration="4.004301923s" totalImagesPullingTime="2.307862459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:55:21.002752096 +0000 UTC m=+650.441012302" watchObservedRunningTime="2026-09-18 18:55:21.004301923 +0000 UTC m=+650.442562127"
Sep 18 18:55:31.995288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:31.995249    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kjc76"
Sep 18 18:55:58.090385 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.090351    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"]
Sep 18 18:55:58.092828 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.092810    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.096320 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.096288    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:55:58.096436 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.096363    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lfvwt\""
Sep 18 18:55:58.096655 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.096632    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:55:58.103027 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.102999    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"]
Sep 18 18:55:58.106020 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.105757    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 18 18:55:58.243872 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.243827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.243872 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.243876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.244074 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.243943    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.244074 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.243960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b3b716-9bc2-4387-a522-cfe713ae0290-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.244144 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.244062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgzts\" (UniqueName: \"kubernetes.io/projected/29b3b716-9bc2-4387-a522-cfe713ae0290-kube-api-access-jgzts\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.244144 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.244111    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.344650 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.344561    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.344650 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.344610    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.344650 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.344651    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.344923 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.344665    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b3b716-9bc2-4387-a522-cfe713ae0290-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.344923 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.344707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgzts\" (UniqueName: \"kubernetes.io/projected/29b3b716-9bc2-4387-a522-cfe713ae0290-kube-api-access-jgzts\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.344923 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.344733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.345071 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.345020    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.345071 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.345043    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.345179 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.345141    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.346943 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.346919    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b3b716-9bc2-4387-a522-cfe713ae0290-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.347099 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.347083    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b3b716-9bc2-4387-a522-cfe713ae0290-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.352424 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.352404    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgzts\" (UniqueName: \"kubernetes.io/projected/29b3b716-9bc2-4387-a522-cfe713ae0290-kube-api-access-jgzts\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq7ck\" (UID: \"29b3b716-9bc2-4387-a522-cfe713ae0290\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.410688 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.410650    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:55:58.680216 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.680181    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"]
Sep 18 18:55:58.681333 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:55:58.681303    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b3b716_9bc2_4387_a522_cfe713ae0290.slice/crio-416edd24e7420e105fc22a25263071f995c591456c7ff280ada43cb75c438cc3 WatchSource:0}: Error finding container 416edd24e7420e105fc22a25263071f995c591456c7ff280ada43cb75c438cc3: Status 404 returned error can't find the container with id 416edd24e7420e105fc22a25263071f995c591456c7ff280ada43cb75c438cc3
Sep 18 18:55:58.683032 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:58.683015    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:55:59.160128 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:55:59.160074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck" event={"ID":"29b3b716-9bc2-4387-a522-cfe713ae0290","Type":"ContainerStarted","Data":"416edd24e7420e105fc22a25263071f995c591456c7ff280ada43cb75c438cc3"}
Sep 18 18:56:00.165822 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:00.165791    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck" event={"ID":"29b3b716-9bc2-4387-a522-cfe713ae0290","Type":"ContainerStarted","Data":"86c9326f461a41e86220a1c0302914d418a38a62efcefe989a06847ea9d89622"}
Sep 18 18:56:00.185988 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:00.185939    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck" podStartSLOduration=1.2450491320000001 podStartE2EDuration="2.185922038s" totalImagesPullingTime="940.872906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:56:00.183983108 +0000 UTC m=+689.622243312" watchObservedRunningTime="2026-09-18 18:56:00.185922038 +0000 UTC m=+689.624182254"
Sep 18 18:56:08.411324 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:08.411289    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:56:08.411720 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:08.411354    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:56:08.423979 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:08.423959    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:56:09.218090 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:09.218063    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq7ck"
Sep 18 18:56:16.685407 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.685365    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"]
Sep 18 18:56:16.687834 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.687817    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.690243 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.690224    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 18 18:56:16.696641 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.696620    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"]
Sep 18 18:56:16.715529 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.715490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.715647 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.715538    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.715647 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.715602    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qxhj\" (UniqueName: \"kubernetes.io/projected/01ce02d2-ee0a-402d-898d-8ab61359389d-kube-api-access-8qxhj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.715727 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.715646    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ce02d2-ee0a-402d-898d-8ab61359389d-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.715727 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.715668    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.715727 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.715692    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.816688 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.816652    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.816688 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.816691    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.816946 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.816736    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qxhj\" (UniqueName: \"kubernetes.io/projected/01ce02d2-ee0a-402d-898d-8ab61359389d-kube-api-access-8qxhj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.816946 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.816762    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ce02d2-ee0a-402d-898d-8ab61359389d-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.816946 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.816784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.816946 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.816809    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.817156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.817093    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.817214 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.817159    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.817263 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.817227    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.819102 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.819076    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ce02d2-ee0a-402d-898d-8ab61359389d-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.819391 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.819368    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ce02d2-ee0a-402d-898d-8ab61359389d-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.828861 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.828839    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qxhj\" (UniqueName: \"kubernetes.io/projected/01ce02d2-ee0a-402d-898d-8ab61359389d-kube-api-access-8qxhj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh\" (UID: \"01ce02d2-ee0a-402d-898d-8ab61359389d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:16.999313 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:16.999277    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:17.272511 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:17.272404    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"]
Sep 18 18:56:17.275172 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:56:17.275133    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ce02d2_ee0a_402d_898d_8ab61359389d.slice/crio-910962bdb545ecb0221e558496f1e419aea0857a809643195025f02864b44682 WatchSource:0}: Error finding container 910962bdb545ecb0221e558496f1e419aea0857a809643195025f02864b44682: Status 404 returned error can't find the container with id 910962bdb545ecb0221e558496f1e419aea0857a809643195025f02864b44682
Sep 18 18:56:18.248901 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:18.248866    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh" event={"ID":"01ce02d2-ee0a-402d-898d-8ab61359389d","Type":"ContainerStarted","Data":"d97aa469744ebd88f9b01afdb3af3466d7d866ba8d17615c4d51409b54a3f762"}
Sep 18 18:56:18.249280 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:18.248916    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh" event={"ID":"01ce02d2-ee0a-402d-898d-8ab61359389d","Type":"ContainerStarted","Data":"910962bdb545ecb0221e558496f1e419aea0857a809643195025f02864b44682"}
Sep 18 18:56:18.265982 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:18.265933    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh" podStartSLOduration=2.059160087 podStartE2EDuration="2.265918497s" totalImagesPullingTime="206.75841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:56:18.264986575 +0000 UTC m=+707.703246816" watchObservedRunningTime="2026-09-18 18:56:18.265918497 +0000 UTC m=+707.704178699"
Sep 18 18:56:27.000264 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:27.000228    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:27.000684 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:27.000293    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:27.013014 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:27.012987    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:27.299145 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:27.299051    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh"
Sep 18 18:56:31.397649 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.397616    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"]
Sep 18 18:56:31.402117 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.402096    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.404768 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.404744    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 18 18:56:31.413161 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.413141    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"]
Sep 18 18:56:31.436824 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.436794    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad654551-c382-4f6a-abe6-accbed87ab01-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.437036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.436840    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.437036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.436887    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.437036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.436934    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.437036 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.436978    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.437204 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.437134    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsfp8\" (UniqueName: \"kubernetes.io/projected/ad654551-c382-4f6a-abe6-accbed87ab01-kube-api-access-tsfp8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538315 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538279    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad654551-c382-4f6a-abe6-accbed87ab01-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538489 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538329    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538489 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538363    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538489 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538489 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538409    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538489 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsfp8\" (UniqueName: \"kubernetes.io/projected/ad654551-c382-4f6a-abe6-accbed87ab01-kube-api-access-tsfp8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538871 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538835    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538999 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538892    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.538999 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.538918    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.540753 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.540737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad654551-c382-4f6a-abe6-accbed87ab01-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.541019 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.541002    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad654551-c382-4f6a-abe6-accbed87ab01-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.547202 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.547181    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsfp8\" (UniqueName: \"kubernetes.io/projected/ad654551-c382-4f6a-abe6-accbed87ab01-kube-api-access-tsfp8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g\" (UID: \"ad654551-c382-4f6a-abe6-accbed87ab01\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.713404 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.713319    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:31.992798 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:31.987314    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"]
Sep 18 18:56:31.994482 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:56:31.994441    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad654551_c382_4f6a_abe6_accbed87ab01.slice/crio-c255437674c3b9241456254d99c5335dc8dd3fd5354125fa3b3ee64ef6c23eda WatchSource:0}: Error finding container c255437674c3b9241456254d99c5335dc8dd3fd5354125fa3b3ee64ef6c23eda: Status 404 returned error can't find the container with id c255437674c3b9241456254d99c5335dc8dd3fd5354125fa3b3ee64ef6c23eda
Sep 18 18:56:32.312856 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:32.312775    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g" event={"ID":"ad654551-c382-4f6a-abe6-accbed87ab01","Type":"ContainerStarted","Data":"070c3b32771f1600c825cf91da09674276ea1055aaf64b47d668d5d7b1f3fbed"}
Sep 18 18:56:32.312856 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:32.312828    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g" event={"ID":"ad654551-c382-4f6a-abe6-accbed87ab01","Type":"ContainerStarted","Data":"c255437674c3b9241456254d99c5335dc8dd3fd5354125fa3b3ee64ef6c23eda"}
Sep 18 18:56:32.331195 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:32.331145    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g" podStartSLOduration=1.143824583 podStartE2EDuration="1.331130017s" totalImagesPullingTime="187.305434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:56:32.328866854 +0000 UTC m=+721.767127052" watchObservedRunningTime="2026-09-18 18:56:32.331130017 +0000 UTC m=+721.769390218"
Sep 18 18:56:41.714292 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:41.714249    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:41.714710 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:41.714333    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:41.726760 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:41.726732    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:56:42.371492 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:56:42.371466    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g"
Sep 18 18:58:47.346205 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.346171    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"]
Sep 18 18:58:47.349850 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.349831    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.358169 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.358148    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-942883-openshift-default-dockercfg-2lcxh\""
Sep 18 18:58:47.374999 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.374968    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"]
Sep 18 18:58:47.447021 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.446989    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-socket\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447188 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447089    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-data\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447188 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447136    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-podinfo\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447285 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447210    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e88335c-687d-4816-b5ca-1a42e08247d0-istiod-ca-cert\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447285 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447240    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-token\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447285 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447267    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-certs\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447391 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447309    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-credential-socket\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447391 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447338    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smk4c\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-kube-api-access-smk4c\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.447391 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.447361    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-envoy\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.548812 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.548777    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e88335c-687d-4816-b5ca-1a42e08247d0-istiod-ca-cert\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549003 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.548824    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-token\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549003 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.548867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-certs\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549003 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.548913    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-credential-socket\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549003 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.548953    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smk4c\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-kube-api-access-smk4c\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549003 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.548980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-envoy\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549272 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.549023    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-socket\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549272 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.549091    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-data\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549272 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.549123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-podinfo\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549596 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.549567    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-certs\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549807 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.549778    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-credential-socket\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.549886 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.549856    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-data\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.550081 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.550053    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e88335c-687d-4816-b5ca-1a42e08247d0-istiod-ca-cert\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.550332 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.550312    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-socket\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.552381 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.552325    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-podinfo\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.552893 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.552865    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-envoy\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.590382 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.589969    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-token\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.591318 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.591280    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smk4c\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-kube-api-access-smk4c\") pod \"e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") " pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.660928 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.660841    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:47.676530 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.676494    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"]
Sep 18 18:58:47.680376 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.680354    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.684742 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.684720    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-15c3ed-openshift-default-dockercfg-qdjsr\""
Sep 18 18:58:47.712678 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.712638    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"]
Sep 18 18:58:47.852718 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.852675    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-credential-socket\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.852901 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.852762    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-certs\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.852901 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.852787    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-data\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.852901 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.852820    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-socket\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.852901 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.852886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12a16a9-f448-4310-bf6e-c82746b28283-istiod-ca-cert\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.853090 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.852946    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-istio-token\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.853090 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.853018    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-envoy\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.853166 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.853128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-499q7\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-kube-api-access-499q7\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.853210 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.853174    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12a16a9-f448-4310-bf6e-c82746b28283-istio-podinfo\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.953972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.953884    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-data\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.953972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.953928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-socket\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.953972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.953950    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12a16a9-f448-4310-bf6e-c82746b28283-istiod-ca-cert\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.953972 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.953971    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-istio-token\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.953993    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-envoy\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954035    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-499q7\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-kube-api-access-499q7\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12a16a9-f448-4310-bf6e-c82746b28283-istio-podinfo\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-credential-socket\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954293 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-certs\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954295    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-data\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954306    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-socket\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.954552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954511    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-certs\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.955007 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954974    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-credential-socket\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.955007 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.954991    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12a16a9-f448-4310-bf6e-c82746b28283-istiod-ca-cert\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.957051 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.957010    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-envoy\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.957502 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.957483    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12a16a9-f448-4310-bf6e-c82746b28283-istio-podinfo\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.958903 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.958885    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"]
Sep 18 18:58:47.961202 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:58:47.961182    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e88335c_687d_4816_b5ca_1a42e08247d0.slice/crio-389bed87fbbf30dd19c34a4408fa81d4a6461495ce0ecaad7a9619dd5b9b2ada WatchSource:0}: Error finding container 389bed87fbbf30dd19c34a4408fa81d4a6461495ce0ecaad7a9619dd5b9b2ada: Status 404 returned error can't find the container with id 389bed87fbbf30dd19c34a4408fa81d4a6461495ce0ecaad7a9619dd5b9b2ada
Sep 18 18:58:47.963340 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.963309    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:58:47.963413 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.963373    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:58:47.963413 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.963404    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:58:47.964785 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.964766    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-499q7\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-kube-api-access-499q7\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.964899 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.964883    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-istio-token\") pod \"e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") " pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:47.991763 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:47.991745    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:48.273995 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.273971    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"]
Sep 18 18:58:48.276471 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:58:48.276438    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12a16a9_f448_4310_bf6e_c82746b28283.slice/crio-588ac6d981ece04d8198139ada1ba8bfef739a66ccfe7f590a6c44325264238a WatchSource:0}: Error finding container 588ac6d981ece04d8198139ada1ba8bfef739a66ccfe7f590a6c44325264238a: Status 404 returned error can't find the container with id 588ac6d981ece04d8198139ada1ba8bfef739a66ccfe7f590a6c44325264238a
Sep 18 18:58:48.278799 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.278768    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:58:48.278869 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.278831    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:58:48.278911 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.278890    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:58:48.961292 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.961251    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" event={"ID":"c12a16a9-f448-4310-bf6e-c82746b28283","Type":"ContainerStarted","Data":"9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"}
Sep 18 18:58:48.961786 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.961316    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" event={"ID":"c12a16a9-f448-4310-bf6e-c82746b28283","Type":"ContainerStarted","Data":"588ac6d981ece04d8198139ada1ba8bfef739a66ccfe7f590a6c44325264238a"}
Sep 18 18:58:48.962745 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.962722    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" event={"ID":"4e88335c-687d-4816-b5ca-1a42e08247d0","Type":"ContainerStarted","Data":"6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"}
Sep 18 18:58:48.962813 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.962771    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" event={"ID":"4e88335c-687d-4816-b5ca-1a42e08247d0","Type":"ContainerStarted","Data":"389bed87fbbf30dd19c34a4408fa81d4a6461495ce0ecaad7a9619dd5b9b2ada"}
Sep 18 18:58:48.992030 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.992006    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:48.995441 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:48.995401    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podStartSLOduration=1.995387364 podStartE2EDuration="1.995387364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:48.99349811 +0000 UTC m=+858.431758315" watchObservedRunningTime="2026-09-18 18:58:48.995387364 +0000 UTC m=+858.433647568"
Sep 18 18:58:49.029716 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:49.029668    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podStartSLOduration=2.029652172 podStartE2EDuration="2.029652172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:49.02930242 +0000 UTC m=+858.467562625" watchObservedRunningTime="2026-09-18 18:58:49.029652172 +0000 UTC m=+858.467912375"
Sep 18 18:58:49.661585 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:49.661550    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:49.666618 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:49.666589    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:49.967925 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:49.967889    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:49.968829 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:49.968810    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:58:49.997157 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:49.997126    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:50.972528 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:50.972495    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:50.973376 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:50.973355    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:58:54.768458 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.768418    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"]
Sep 18 18:58:54.773365 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.773344    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.795104 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.795078    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-15c3ed\""
Sep 18 18:58:54.795255 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.795139    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-15c3ed-dockercfg-z284l\""
Sep 18 18:58:54.818604 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.818573    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82ntr\" (UniqueName: \"kubernetes.io/projected/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-kube-api-access-82ntr\") pod \"payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.818780 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.818758    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.826343 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.826312    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"]
Sep 18 18:58:54.919549 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.919493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.919726 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.919574    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82ntr\" (UniqueName: \"kubernetes.io/projected/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-kube-api-access-82ntr\") pod \"payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.920187 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.920166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.942785 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.942759    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82ntr\" (UniqueName: \"kubernetes.io/projected/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-kube-api-access-82ntr\") pod \"payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:54.963824 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.963791    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"]
Sep 18 18:58:54.967453 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:54.967435    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.003852 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.003824    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"]
Sep 18 18:58:55.020240 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.020170    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7znj\" (UniqueName: \"kubernetes.io/projected/a060e899-ee44-4687-9d30-e28f0be7a279-kube-api-access-p7znj\") pod \"payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") " pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.020240 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.020223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a060e899-ee44-4687-9d30-e28f0be7a279-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") " pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.083671 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.083639    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:55.121790 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.121157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7znj\" (UniqueName: \"kubernetes.io/projected/a060e899-ee44-4687-9d30-e28f0be7a279-kube-api-access-p7znj\") pod \"payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") " pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.121790 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.121266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a060e899-ee44-4687-9d30-e28f0be7a279-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") " pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.122231 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.122209    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a060e899-ee44-4687-9d30-e28f0be7a279-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") " pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.142346 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.141979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7znj\" (UniqueName: \"kubernetes.io/projected/a060e899-ee44-4687-9d30-e28f0be7a279-kube-api-access-p7znj\") pod \"payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") " pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.278312 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.278228    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.449257 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.449091    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"]
Sep 18 18:58:55.451593 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:58:55.451568    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4850a56b_3199_4b8a_af1d_1c1e4e3d5b23.slice/crio-6ff8329cbfa70800852915170f368b0a071541f33347b2d0220d95759ab77de1 WatchSource:0}: Error finding container 6ff8329cbfa70800852915170f368b0a071541f33347b2d0220d95759ab77de1: Status 404 returned error can't find the container with id 6ff8329cbfa70800852915170f368b0a071541f33347b2d0220d95759ab77de1
Sep 18 18:58:55.602572 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.602477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"]
Sep 18 18:58:55.608904 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:58:55.608874    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda060e899_ee44_4687_9d30_e28f0be7a279.slice/crio-3b6926f3ae537aa5cc3a343bfe004608a8dcf22d71c333eab9025bc240a844f9 WatchSource:0}: Error finding container 3b6926f3ae537aa5cc3a343bfe004608a8dcf22d71c333eab9025bc240a844f9: Status 404 returned error can't find the container with id 3b6926f3ae537aa5cc3a343bfe004608a8dcf22d71c333eab9025bc240a844f9
Sep 18 18:58:55.998095 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.998060    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k" event={"ID":"a060e899-ee44-4687-9d30-e28f0be7a279","Type":"ContainerStarted","Data":"93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990"}
Sep 18 18:58:55.998567 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.998120    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k" event={"ID":"a060e899-ee44-4687-9d30-e28f0be7a279","Type":"ContainerStarted","Data":"3b6926f3ae537aa5cc3a343bfe004608a8dcf22d71c333eab9025bc240a844f9"}
Sep 18 18:58:55.998567 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.998164    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:58:55.999727 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.999704    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x" event={"ID":"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23","Type":"ContainerStarted","Data":"253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd"}
Sep 18 18:58:55.999823 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.999752    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:58:55.999823 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:55.999770    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x" event={"ID":"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23","Type":"ContainerStarted","Data":"6ff8329cbfa70800852915170f368b0a071541f33347b2d0220d95759ab77de1"}
Sep 18 18:58:56.036809 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:56.036755    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k" podStartSLOduration=2.036739016 podStartE2EDuration="2.036739016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:56.035365449 +0000 UTC m=+865.473625657" watchObservedRunningTime="2026-09-18 18:58:56.036739016 +0000 UTC m=+865.474999211"
Sep 18 18:58:56.074046 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:56.073996    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x" podStartSLOduration=2.073982007 podStartE2EDuration="2.073982007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:56.073556386 +0000 UTC m=+865.511816591" watchObservedRunningTime="2026-09-18 18:58:56.073982007 +0000 UTC m=+865.512242211"
Sep 18 18:58:56.970185 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:56.970144    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"]
Sep 18 18:58:56.974373 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:56.974356    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:56.986935 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:56.986916    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-942883\""
Sep 18 18:58:56.990838 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:56.990815    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-942883-dockercfg-vfs9q\""
Sep 18 18:58:57.009437 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.009413    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"]
Sep 18 18:58:57.040161 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.040131    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20e186dc-5701-4850-9d39-c07b61f199d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.040289 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.040174    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m58gl\" (UniqueName: \"kubernetes.io/projected/20e186dc-5701-4850-9d39-c07b61f199d3-kube-api-access-m58gl\") pod \"payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.141740 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.141698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20e186dc-5701-4850-9d39-c07b61f199d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.141945 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.141766    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m58gl\" (UniqueName: \"kubernetes.io/projected/20e186dc-5701-4850-9d39-c07b61f199d3-kube-api-access-m58gl\") pod \"payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.142403 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.142381    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20e186dc-5701-4850-9d39-c07b61f199d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.161093 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.161062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m58gl\" (UniqueName: \"kubernetes.io/projected/20e186dc-5701-4850-9d39-c07b61f199d3-kube-api-access-m58gl\") pod \"payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.285475 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.285397    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:57.593622 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:58:57.593591    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e186dc_5701_4850_9d39_c07b61f199d3.slice/crio-3926a6433e1add1097cfb742a8f8a14ab0064ac906aec931bc91f68f675a315a WatchSource:0}: Error finding container 3926a6433e1add1097cfb742a8f8a14ab0064ac906aec931bc91f68f675a315a: Status 404 returned error can't find the container with id 3926a6433e1add1097cfb742a8f8a14ab0064ac906aec931bc91f68f675a315a
Sep 18 18:58:57.664159 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:57.664130    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"]
Sep 18 18:58:58.014782 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:58.014741    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll" event={"ID":"20e186dc-5701-4850-9d39-c07b61f199d3","Type":"ContainerStarted","Data":"95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f"}
Sep 18 18:58:58.015201 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:58.014795    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll" event={"ID":"20e186dc-5701-4850-9d39-c07b61f199d3","Type":"ContainerStarted","Data":"3926a6433e1add1097cfb742a8f8a14ab0064ac906aec931bc91f68f675a315a"}
Sep 18 18:58:58.015201 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:58.014817    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:58:58.060369 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:58.060319    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll" podStartSLOduration=2.06030493 podStartE2EDuration="2.06030493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:58:58.054440177 +0000 UTC m=+867.492700381" watchObservedRunningTime="2026-09-18 18:58:58.06030493 +0000 UTC m=+867.498565134"
Sep 18 18:58:59.183712 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.183678    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"]
Sep 18 18:58:59.187164 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.187146    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.197095 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.197070    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-1a444c-dockercfg-jrf8b\""
Sep 18 18:58:59.198621 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.198600    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-1a444c\""
Sep 18 18:58:59.215539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.215497    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"]
Sep 18 18:58:59.266834 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.266793    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.266834 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.266849    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bj8m\" (UniqueName: \"kubernetes.io/projected/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-kube-api-access-7bj8m\") pod \"payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.368350 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.368317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.368350 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.368356    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bj8m\" (UniqueName: \"kubernetes.io/projected/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-kube-api-access-7bj8m\") pod \"payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.369065 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.369043    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.384080 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.384046    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bj8m\" (UniqueName: \"kubernetes.io/projected/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-kube-api-access-7bj8m\") pod \"payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.498231 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.498203    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:58:59.803184 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:58:59.803146    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"]
Sep 18 18:58:59.804885 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:58:59.804858    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b84adb_d4de_498f_bcc5_13ad6ac6939f.slice/crio-474e149678dc451c16a62e1959ddff8e8e39cc300cf9e3b53ca5fc6594adf3ba WatchSource:0}: Error finding container 474e149678dc451c16a62e1959ddff8e8e39cc300cf9e3b53ca5fc6594adf3ba: Status 404 returned error can't find the container with id 474e149678dc451c16a62e1959ddff8e8e39cc300cf9e3b53ca5fc6594adf3ba
Sep 18 18:59:00.025862 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:00.025825    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5" event={"ID":"c7b84adb-d4de-498f-bcc5-13ad6ac6939f","Type":"ContainerStarted","Data":"406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7"}
Sep 18 18:59:00.026066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:00.026004    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:59:00.026066 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:00.026024    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5" event={"ID":"c7b84adb-d4de-498f-bcc5-13ad6ac6939f","Type":"ContainerStarted","Data":"474e149678dc451c16a62e1959ddff8e8e39cc300cf9e3b53ca5fc6594adf3ba"}
Sep 18 18:59:06.876083 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.875990    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5" podStartSLOduration=7.87597184 podStartE2EDuration="7.87597184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:00.132738517 +0000 UTC m=+869.570998733" watchObservedRunningTime="2026-09-18 18:59:06.87597184 +0000 UTC m=+876.314232044"
Sep 18 18:59:06.876948 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.876929    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"]
Sep 18 18:59:06.880862 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.880843    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.889788 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.889770    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3a81e5-openshift-default-dockercfg-srjlx\""
Sep 18 18:59:06.932501 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.932414    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"]
Sep 18 18:59:06.968354 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-socket\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968562 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968401    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istiod-ca-cert\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968562 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968445    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-credential-socket\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968562 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968493    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-envoy\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968737 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968575    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-certs\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968737 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968635    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn2kw\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-kube-api-access-vn2kw\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968737 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-token\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968884 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968737    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-data\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:06.968884 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:06.968798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-podinfo\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.008682 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.008654    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 18:59:07.008841 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.008735    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 18:59:07.069723 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.069688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-podinfo\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.069723 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.069725    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-socket\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.069950 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.069849    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istiod-ca-cert\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.069950 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.069888    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-credential-socket\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.069950 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.069945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-envoy\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070107 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070016    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-certs\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070107 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2kw\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-kube-api-access-vn2kw\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070107 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-token\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070252 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070151    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-data\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070252 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070161    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-socket\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070396 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070369    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-credential-socket\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070637 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-data\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070810 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070647    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-certs\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.070810 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.070667    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istiod-ca-cert\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.072275 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.072254    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-envoy\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.072380 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.072362    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-podinfo\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.094877 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.094848    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2kw\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-kube-api-access-vn2kw\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.100013 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.099989    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-token\") pod \"e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") " pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.192156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.192076    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:07.338552 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.337435    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"]
Sep 18 18:59:07.357430 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.357118    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.406374 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.406340    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-fb8843\""
Sep 18 18:59:07.474694 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.474652    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0571f833-7e15-420b-8dbf-221454fd4aa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.474957 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.474933    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxw7q\" (UniqueName: \"kubernetes.io/projected/0571f833-7e15-420b-8dbf-221454fd4aa0-kube-api-access-xxw7q\") pod \"payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.496158 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.496125    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"]
Sep 18 18:59:07.576665 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.576614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0571f833-7e15-420b-8dbf-221454fd4aa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.576956 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.576929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxw7q\" (UniqueName: \"kubernetes.io/projected/0571f833-7e15-420b-8dbf-221454fd4aa0-kube-api-access-xxw7q\") pod \"payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.577316 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.577296    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0571f833-7e15-420b-8dbf-221454fd4aa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.604839 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.604808    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxw7q\" (UniqueName: \"kubernetes.io/projected/0571f833-7e15-420b-8dbf-221454fd4aa0-kube-api-access-xxw7q\") pod \"payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.630865 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.630834    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"]
Sep 18 18:59:07.634933 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.634908    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.639303 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.639281    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-fb8843-dockercfg-gbnm9\""
Sep 18 18:59:07.668978 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.668953    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:07.686117 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.686084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"]
Sep 18 18:59:07.688293 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:59:07.688262    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de4ebd3_e89f_46f9_bcb2_861ca1cbdb67.slice/crio-3a98b6509fb64c0d9ea5572beae67e09e53db3c1154f8394cb7d57169fd87aa4 WatchSource:0}: Error finding container 3a98b6509fb64c0d9ea5572beae67e09e53db3c1154f8394cb7d57169fd87aa4: Status 404 returned error can't find the container with id 3a98b6509fb64c0d9ea5572beae67e09e53db3c1154f8394cb7d57169fd87aa4
Sep 18 18:59:07.691366 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.691330    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:07.691476 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.691407    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:07.691476 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.691439    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:07.728421 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.728390    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"]
Sep 18 18:59:07.780584 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.779634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90d57164-6148-4db1-8c41-84635e0a2c31-plugins-config-volume\") pod \"payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") " pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.780584 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.779831    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vfps\" (UniqueName: \"kubernetes.io/projected/90d57164-6148-4db1-8c41-84635e0a2c31-kube-api-access-4vfps\") pod \"payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") " pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.881327 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.881221    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vfps\" (UniqueName: \"kubernetes.io/projected/90d57164-6148-4db1-8c41-84635e0a2c31-kube-api-access-4vfps\") pod \"payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") " pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.882531 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.882480    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90d57164-6148-4db1-8c41-84635e0a2c31-plugins-config-volume\") pod \"payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") " pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.883529 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.883490    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90d57164-6148-4db1-8c41-84635e0a2c31-plugins-config-volume\") pod \"payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") " pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.897101 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.897075    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vfps\" (UniqueName: \"kubernetes.io/projected/90d57164-6148-4db1-8c41-84635e0a2c31-kube-api-access-4vfps\") pod \"payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") " pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:07.957372 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:07.957281    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:08.036814 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.036758    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"]
Sep 18 18:59:08.117615 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.117081    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" event={"ID":"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67","Type":"ContainerStarted","Data":"2208783ba84dafbd176bd7831a999b3f179727dba3a9bb735cb30d8235a48515"}
Sep 18 18:59:08.117615 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.117143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" event={"ID":"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67","Type":"ContainerStarted","Data":"3a98b6509fb64c0d9ea5572beae67e09e53db3c1154f8394cb7d57169fd87aa4"}
Sep 18 18:59:08.119341 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.119297    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk" event={"ID":"0571f833-7e15-420b-8dbf-221454fd4aa0","Type":"ContainerStarted","Data":"401d38ed01035e890047ce3dd490048a3a89be47e39e991104f365551c43be7d"}
Sep 18 18:59:08.192869 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.192785    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:08.194763 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.194718    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 18:59:08.194976 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.194777    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 18:59:08.204947 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.204742    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podStartSLOduration=2.204718382 podStartE2EDuration="2.204718382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:08.204618147 +0000 UTC m=+877.642878352" watchObservedRunningTime="2026-09-18 18:59:08.204718382 +0000 UTC m=+877.642978590"
Sep 18 18:59:08.311581 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:08.311476    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"]
Sep 18 18:59:08.316235 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:59:08.316153    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d57164_6148_4db1_8c41_84635e0a2c31.slice/crio-be4dab378f5b28c1bf078f568ddfa5eaa959b56e5cc18cddf59cfd1b294abd87 WatchSource:0}: Error finding container be4dab378f5b28c1bf078f568ddfa5eaa959b56e5cc18cddf59cfd1b294abd87: Status 404 returned error can't find the container with id be4dab378f5b28c1bf078f568ddfa5eaa959b56e5cc18cddf59cfd1b294abd87
Sep 18 18:59:09.022485 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.022449    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 18:59:09.129274 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.129225    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq" event={"ID":"90d57164-6148-4db1-8c41-84635e0a2c31","Type":"ContainerStarted","Data":"b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb"}
Sep 18 18:59:09.129464 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.129296    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq" event={"ID":"90d57164-6148-4db1-8c41-84635e0a2c31","Type":"ContainerStarted","Data":"be4dab378f5b28c1bf078f568ddfa5eaa959b56e5cc18cddf59cfd1b294abd87"}
Sep 18 18:59:09.129560 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.129502    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:09.132405 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.132378    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk" event={"ID":"0571f833-7e15-420b-8dbf-221454fd4aa0","Type":"ContainerStarted","Data":"93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd"}
Sep 18 18:59:09.132547 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.132424    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:09.197675 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.197645    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:09.219106 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:09.219041    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq" podStartSLOduration=2.219020154 podStartE2EDuration="2.219020154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:09.217742305 +0000 UTC m=+878.656002510" watchObservedRunningTime="2026-09-18 18:59:09.219020154 +0000 UTC m=+878.657280360"
Sep 18 18:59:10.140819 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:10.140790    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:10.142574 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:10.142550    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:10.231091 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:10.231028    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk" podStartSLOduration=3.23100612 podStartE2EDuration="3.23100612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:09.386489154 +0000 UTC m=+878.824749357" watchObservedRunningTime="2026-09-18 18:59:10.23100612 +0000 UTC m=+879.669266328"
Sep 18 18:59:11.036245 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.036218    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 18:59:11.147365 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.147337    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerID="6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6" exitCode=0
Sep 18 18:59:11.147745 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.147407    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" event={"ID":"4e88335c-687d-4816-b5ca-1a42e08247d0","Type":"ContainerDied","Data":"6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"}
Sep 18 18:59:11.147983 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.147960    2568 scope.go:117] "RemoveContainer" containerID="6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"
Sep 18 18:59:11.149942 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.149910    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:11.150283 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.149997    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:11.150283 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.150046    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:11.150765 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.150742    2568 generic.go:356] "Generic (PLEG): container finished" podID="c12a16a9-f448-4310-bf6e-c82746b28283" containerID="9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6" exitCode=0
Sep 18 18:59:11.150883 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.150814    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" event={"ID":"c12a16a9-f448-4310-bf6e-c82746b28283","Type":"ContainerDied","Data":"9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"}
Sep 18 18:59:11.151435 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.151373    2568 scope.go:117] "RemoveContainer" containerID="9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"
Sep 18 18:59:11.152660 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.152626    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:11.152759 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.152706    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:11.152759 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:11.152749    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:12.158270 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.158239    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" event={"ID":"4e88335c-687d-4816-b5ca-1a42e08247d0","Type":"ContainerStarted","Data":"e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974"}
Sep 18 18:59:12.160344 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.160321    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" event={"ID":"c12a16a9-f448-4310-bf6e-c82746b28283","Type":"ContainerStarted","Data":"4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32"}
Sep 18 18:59:12.662080 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.662037    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:59:12.662291 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.662206    2568 patch_prober.go:28] interesting pod/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 18:59:12.662291 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.662250    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 18:59:12.992395 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.992361    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:59:12.992644 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.992619    2568 patch_prober.go:28] interesting pod/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:59:12.992761 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:12.992668    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 18 18:59:13.662412 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:13.662371    2568 patch_prober.go:28] interesting pod/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 18:59:13.662931 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:13.662434    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 18:59:13.992741 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:13.992706    2568 patch_prober.go:28] interesting pod/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:59:13.992909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:13.992759    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 18 18:59:14.663548 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:14.662929    2568 patch_prober.go:28] interesting pod/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 18:59:14.663548 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:14.662988    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 18:59:14.992848 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:14.992812    2568 patch_prober.go:28] interesting pod/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:59:14.993031 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:14.992872    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 18 18:59:15.662233 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:15.662186    2568 patch_prober.go:28] interesting pod/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 18:59:15.662454 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:15.662274    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 18:59:15.992211 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:15.992175    2568 patch_prober.go:28] interesting pod/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:59:15.992629 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:15.992231    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 18 18:59:16.662540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:16.661960    2568 patch_prober.go:28] interesting pod/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 18:59:16.662540 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:16.662026    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 18:59:16.993018 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:16.992984    2568 patch_prober.go:28] interesting pod/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:59:16.993398 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:16.993045    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 18 18:59:17.661723 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:17.661676    2568 patch_prober.go:28] interesting pod/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 18:59:17.661982 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:17.661748    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 18:59:17.992542 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:17.992478    2568 patch_prober.go:28] interesting pod/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 18 18:59:17.992727 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:17.992571    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 18 18:59:18.686228 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:18.686196    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:59:18.688773 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:18.688744    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:59:18.690899 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:18.690151    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 18:59:18.996689 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:18.996657    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:59:18.996970 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:18.996950    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:59:18.997618 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:18.997598    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 18:59:20.142178 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:20.142149    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 18:59:20.142553 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:20.142214    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 18:59:31.005825 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:31.005794    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:59:31.008199 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:31.008169    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 18:59:32.753022 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.752988    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"]
Sep 18 18:59:32.759803 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.759772    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:32.763770 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.763747    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 18:59:32.764148 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.764126    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:59:32.781082 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.781053    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"]
Sep 18 18:59:32.859999 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.859968    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a054abb0-8b5e-4e63-a7cf-dfdad436773c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj\" (UID: \"a054abb0-8b5e-4e63-a7cf-dfdad436773c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:32.960800 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.960760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a054abb0-8b5e-4e63-a7cf-dfdad436773c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj\" (UID: \"a054abb0-8b5e-4e63-a7cf-dfdad436773c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:32.961778 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:32.961748    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a054abb0-8b5e-4e63-a7cf-dfdad436773c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj\" (UID: \"a054abb0-8b5e-4e63-a7cf-dfdad436773c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:33.071746 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:33.071660    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:33.356771 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:33.356746    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"]
Sep 18 18:59:33.358267 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:59:33.358241    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda054abb0_8b5e_4e63_a7cf_dfdad436773c.slice/crio-c43aa9a860fb2403b0635b1eebc6f1f4b8a90a235cb7c8de4c7031c4b85aefa7 WatchSource:0}: Error finding container c43aa9a860fb2403b0635b1eebc6f1f4b8a90a235cb7c8de4c7031c4b85aefa7: Status 404 returned error can't find the container with id c43aa9a860fb2403b0635b1eebc6f1f4b8a90a235cb7c8de4c7031c4b85aefa7
Sep 18 18:59:34.290917 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:34.290874    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj" event={"ID":"a054abb0-8b5e-4e63-a7cf-dfdad436773c","Type":"ContainerStarted","Data":"c43aa9a860fb2403b0635b1eebc6f1f4b8a90a235cb7c8de4c7031c4b85aefa7"}
Sep 18 18:59:36.304611 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:36.304495    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj" event={"ID":"a054abb0-8b5e-4e63-a7cf-dfdad436773c","Type":"ContainerStarted","Data":"6a251e02b935bdf09f1fc199f338d49155ba0e633019e70444561723d87af903"}
Sep 18 18:59:36.336777 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:36.336725    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj" podStartSLOduration=1.98266462 podStartE2EDuration="4.336710593s" totalImagesPullingTime="2.354045973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:36.334696391 +0000 UTC m=+905.772956595" watchObservedRunningTime="2026-09-18 18:59:36.336710593 +0000 UTC m=+905.774970796"
Sep 18 18:59:37.311733 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:37.311697    2568 generic.go:356] "Generic (PLEG): container finished" podID="a054abb0-8b5e-4e63-a7cf-dfdad436773c" containerID="6a251e02b935bdf09f1fc199f338d49155ba0e633019e70444561723d87af903" exitCode=0
Sep 18 18:59:37.312110 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:37.311727    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj" event={"ID":"a054abb0-8b5e-4e63-a7cf-dfdad436773c","Type":"ContainerDied","Data":"6a251e02b935bdf09f1fc199f338d49155ba0e633019e70444561723d87af903"}
Sep 18 18:59:38.510801 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:38.510779    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:38.635677 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:38.635060    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a054abb0-8b5e-4e63-a7cf-dfdad436773c-maas-api-service-ca\") pod \"a054abb0-8b5e-4e63-a7cf-dfdad436773c\" (UID: \"a054abb0-8b5e-4e63-a7cf-dfdad436773c\") "
Sep 18 18:59:38.636037 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:38.635997    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a054abb0-8b5e-4e63-a7cf-dfdad436773c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a054abb0-8b5e-4e63-a7cf-dfdad436773c" (UID: "a054abb0-8b5e-4e63-a7cf-dfdad436773c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:59:38.746539 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:38.746295    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a054abb0-8b5e-4e63-a7cf-dfdad436773c-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 18:59:39.331030 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:39.330870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj" event={"ID":"a054abb0-8b5e-4e63-a7cf-dfdad436773c","Type":"ContainerDied","Data":"c43aa9a860fb2403b0635b1eebc6f1f4b8a90a235cb7c8de4c7031c4b85aefa7"}
Sep 18 18:59:39.331030 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:39.330916    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"
Sep 18 18:59:39.331030 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:39.330939    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c43aa9a860fb2403b0635b1eebc6f1f4b8a90a235cb7c8de4c7031c4b85aefa7"
Sep 18 18:59:45.478488 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:45.478445    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"]
Sep 18 18:59:45.482745 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:45.482714    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cbeacf3-gj5nj"]
Sep 18 18:59:46.783872 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.783839    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"]
Sep 18 18:59:46.785325 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.784296    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a054abb0-8b5e-4e63-a7cf-dfdad436773c" containerName="revoke-keys"
Sep 18 18:59:46.785325 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.784309    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a054abb0-8b5e-4e63-a7cf-dfdad436773c" containerName="revoke-keys"
Sep 18 18:59:46.785325 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.784375    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a054abb0-8b5e-4e63-a7cf-dfdad436773c" containerName="revoke-keys"
Sep 18 18:59:46.786544 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.786503    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.788953 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.788933    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f4465858-gw-openshift-default-dockercfg-5vpg4\""
Sep 18 18:59:46.809501 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.809464    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"]
Sep 18 18:59:46.825747 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.825718    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r97g\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-kube-api-access-5r97g\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.825909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.825753    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-socket\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.825909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.825780    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-data\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.825909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.825831    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-podinfo\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.825909 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.825859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c220890-39c8-4869-9c25-3b9027a7b3ee-istiod-ca-cert\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.826051 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.825974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-certs\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.826051 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.826033    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-token\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.826115 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.826090    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-credential-socket\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.826149 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.826118    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-envoy\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927596 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927556    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-credential-socket\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927769 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927616    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-envoy\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927769 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927662    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r97g\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-kube-api-access-5r97g\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927769 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927682    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-socket\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927954 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927826    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-data\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927954 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927884    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-podinfo\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.927954 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.927920    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c220890-39c8-4869-9c25-3b9027a7b3ee-istiod-ca-cert\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928178 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928019    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-credential-socket\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928178 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-socket\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928178 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-certs\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928178 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928143    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-token\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928406 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928239    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-data\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928406 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928335    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-certs\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.928732 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.928708    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c220890-39c8-4869-9c25-3b9027a7b3ee-istiod-ca-cert\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.929966 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.929943    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-envoy\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.930461 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.930444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-podinfo\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.939542 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.939491    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-token\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:46.941507 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:46.941490    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r97g\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-kube-api-access-5r97g\") pod \"e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") " pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:47.055454 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.055301    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a054abb0-8b5e-4e63-a7cf-dfdad436773c" path="/var/lib/kubelet/pods/a054abb0-8b5e-4e63-a7cf-dfdad436773c/volumes"
Sep 18 18:59:47.097874 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.097832    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:47.387654 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.387567    2568 generic.go:356] "Generic (PLEG): container finished" podID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerID="2208783ba84dafbd176bd7831a999b3f179727dba3a9bb735cb30d8235a48515" exitCode=0
Sep 18 18:59:47.387654 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.387645    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" event={"ID":"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67","Type":"ContainerDied","Data":"2208783ba84dafbd176bd7831a999b3f179727dba3a9bb735cb30d8235a48515"}
Sep 18 18:59:47.388085 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.388071    2568 scope.go:117] "RemoveContainer" containerID="2208783ba84dafbd176bd7831a999b3f179727dba3a9bb735cb30d8235a48515"
Sep 18 18:59:47.389341 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.389311    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:47.389418 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.389379    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:47.389418 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.389410    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:47.400124 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.400102    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"]
Sep 18 18:59:47.401385 ip-10-0-132-94 kubenswrapper[2568]: W0918 18:59:47.401363    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c220890_39c8_4869_9c25_3b9027a7b3ee.slice/crio-07dee786477cc1ff63b099f67a0ee4731988061cfa7a58b31de92c3ed20371ec WatchSource:0}: Error finding container 07dee786477cc1ff63b099f67a0ee4731988061cfa7a58b31de92c3ed20371ec: Status 404 returned error can't find the container with id 07dee786477cc1ff63b099f67a0ee4731988061cfa7a58b31de92c3ed20371ec
Sep 18 18:59:47.403301 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.403272    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:47.403379 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.403331    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:47.403379 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:47.403361    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:59:48.395255 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:48.395214    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" event={"ID":"2c220890-39c8-4869-9c25-3b9027a7b3ee","Type":"ContainerStarted","Data":"117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"}
Sep 18 18:59:48.395718 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:48.395279    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" event={"ID":"2c220890-39c8-4869-9c25-3b9027a7b3ee","Type":"ContainerStarted","Data":"07dee786477cc1ff63b099f67a0ee4731988061cfa7a58b31de92c3ed20371ec"}
Sep 18 18:59:48.397156 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:48.397136    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" event={"ID":"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67","Type":"ContainerStarted","Data":"e5fa712e41c0678341bab2deb458f675a9545a032d95c9fece9c17283cde9a80"}
Sep 18 18:59:48.454017 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:48.453959    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podStartSLOduration=2.453941441 podStartE2EDuration="2.453941441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:59:48.452415548 +0000 UTC m=+917.890675752" watchObservedRunningTime="2026-09-18 18:59:48.453941441 +0000 UTC m=+917.892201645"
Sep 18 18:59:49.098298 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.098258    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:49.103614 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.103588    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:49.193094 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.193058    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:49.193363 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.193342    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 18:59:49.193409 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.193389    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 18:59:49.403057 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.402966    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:49.403879 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:49.403859    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 18:59:50.193288 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:50.193250    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 18:59:50.193467 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:50.193317    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 18:59:51.192497 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:51.192460    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 18:59:51.192971 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:51.192533    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 18:59:52.193284 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:52.193247    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 18:59:52.193738 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:52.193328    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 18:59:53.193043 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:53.193008    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 18:59:53.193216 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:53.193062    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 18:59:54.196760 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:54.196722    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:54.197135 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:54.196986    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 18:59:54.197591 ip-10-0-132-94 kubenswrapper[2568]: I0918 18:59:54.197569    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 19:00:00.287027 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.286991    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"]
Sep 18 19:00:00.290936 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.290913    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"]
Sep 18 19:00:00.291079 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.291028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"
Sep 18 19:00:00.294625 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.294603    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"]
Sep 18 19:00:00.294746 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.294722    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"
Sep 18 19:00:00.298043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.298021    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"
Sep 18 19:00:00.314508 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.314479    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:00:00.315708 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.315694    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"
Sep 18 19:00:00.320585 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.320565    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"
Sep 18 19:00:00.322830 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.322814    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"
Sep 18 19:00:00.374955 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.374929    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"]
Sep 18 19:00:00.383895 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.383404    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"]
Sep 18 19:00:00.383895 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.383843    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"]
Sep 18 19:00:00.389812 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.389380    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"
Sep 18 19:00:00.406944 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.406904    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"]
Sep 18 19:00:00.408375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.408098    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"
Sep 18 19:00:00.415798 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.415774    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"]
Sep 18 19:00:00.712427 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.712389    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"]
Sep 18 19:00:00.730540 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.730490    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"]
Sep 18 19:00:00.736294 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:00:00.736267    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73464364_b803_4183_a5fa_d2e0b68e4b64.slice/crio-ecff38f3a7b63e2d3c89f2e678810f080197f2128b21aa512a048c17d6cbc4bb WatchSource:0}: Error finding container ecff38f3a7b63e2d3c89f2e678810f080197f2128b21aa512a048c17d6cbc4bb: Status 404 returned error can't find the container with id ecff38f3a7b63e2d3c89f2e678810f080197f2128b21aa512a048c17d6cbc4bb
Sep 18 19:00:00.736833 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:00:00.736759    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1be9f1_6611_4421_a5ab_96dbc0d13bee.slice/crio-0bd71d327a583421e55f9f9575d4382bf4854aa53df6e277494b721f8407530a WatchSource:0}: Error finding container 0bd71d327a583421e55f9f9575d4382bf4854aa53df6e277494b721f8407530a: Status 404 returned error can't find the container with id 0bd71d327a583421e55f9f9575d4382bf4854aa53df6e277494b721f8407530a
Sep 18 19:00:00.774940 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.774914    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"]
Sep 18 19:00:00.775772 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:00:00.775754    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63996dd5_a52f_4d8c_9e37_ef1c2e4f2d4e.slice/crio-43912f8a6540463b8f528b5e539223be3d254f921861540e3b415be9904a4553 WatchSource:0}: Error finding container 43912f8a6540463b8f528b5e539223be3d254f921861540e3b415be9904a4553: Status 404 returned error can't find the container with id 43912f8a6540463b8f528b5e539223be3d254f921861540e3b415be9904a4553
Sep 18 19:00:00.857478 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:00.857452    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"]
Sep 18 19:00:00.926311 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:00:00.926279    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a53f7b_9b25_4866_94d5_3316689d7d73.slice/crio-52364fba4abc7ef5f06097afdbeb8ef0abdd753ca846453859a68060a7d2e3e8 WatchSource:0}: Error finding container 52364fba4abc7ef5f06097afdbeb8ef0abdd753ca846453859a68060a7d2e3e8: Status 404 returned error can't find the container with id 52364fba4abc7ef5f06097afdbeb8ef0abdd753ca846453859a68060a7d2e3e8
Sep 18 19:00:01.478490 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.478456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh" event={"ID":"a5a53f7b-9b25-4866-94d5-3316689d7d73","Type":"ContainerStarted","Data":"37842cc2b671f31972b6d11b4ede9898f7b6b730516549588f07dc5662d26166"}
Sep 18 19:00:01.478942 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.478531    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh" event={"ID":"a5a53f7b-9b25-4866-94d5-3316689d7d73","Type":"ContainerStarted","Data":"52364fba4abc7ef5f06097afdbeb8ef0abdd753ca846453859a68060a7d2e3e8"}
Sep 18 19:00:01.480113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.480088    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42" event={"ID":"63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e","Type":"ContainerStarted","Data":"041d79e705ecfb23eda5f9834e90a8d0f8cb9a3afaf6887d428268059e1028af"}
Sep 18 19:00:01.480256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.480132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42" event={"ID":"63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e","Type":"ContainerStarted","Data":"43912f8a6540463b8f528b5e539223be3d254f921861540e3b415be9904a4553"}
Sep 18 19:00:01.481564 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.481540    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn" event={"ID":"cc1be9f1-6611-4421-a5ab-96dbc0d13bee","Type":"ContainerStarted","Data":"59610dea0d40b97c62debfa432df742e4460837b40f2027cff7558e246f3b4df"}
Sep 18 19:00:01.481645 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.481580    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn" event={"ID":"cc1be9f1-6611-4421-a5ab-96dbc0d13bee","Type":"ContainerStarted","Data":"0bd71d327a583421e55f9f9575d4382bf4854aa53df6e277494b721f8407530a"}
Sep 18 19:00:01.482881 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.482863    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h" event={"ID":"73464364-b803-4183-a5fa-d2e0b68e4b64","Type":"ContainerStarted","Data":"15fa15a049a8163c28384871f315a9c3980a70262be5cb79cec4291f57b18546"}
Sep 18 19:00:01.482938 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.482894    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h" event={"ID":"73464364-b803-4183-a5fa-d2e0b68e4b64","Type":"ContainerStarted","Data":"ecff38f3a7b63e2d3c89f2e678810f080197f2128b21aa512a048c17d6cbc4bb"}
Sep 18 19:00:01.546120 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.546063    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh" podStartSLOduration=1.546044803 podStartE2EDuration="1.546044803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:01.545398263 +0000 UTC m=+930.983658478" watchObservedRunningTime="2026-09-18 19:00:01.546044803 +0000 UTC m=+930.984305004"
Sep 18 19:00:01.608316 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.608257    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn" podStartSLOduration=1.608239765 podStartE2EDuration="1.608239765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:01.606908938 +0000 UTC m=+931.045169141" watchObservedRunningTime="2026-09-18 19:00:01.608239765 +0000 UTC m=+931.046499969"
Sep 18 19:00:01.694247 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:01.694190    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42" podStartSLOduration=1.694174377 podStartE2EDuration="1.694174377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:01.678879645 +0000 UTC m=+931.117139846" watchObservedRunningTime="2026-09-18 19:00:01.694174377 +0000 UTC m=+931.132434635"
Sep 18 19:00:02.490013 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.489980    2568 generic.go:356] "Generic (PLEG): container finished" podID="63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e" containerID="041d79e705ecfb23eda5f9834e90a8d0f8cb9a3afaf6887d428268059e1028af" exitCode=0
Sep 18 19:00:02.490385 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.490066    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42" event={"ID":"63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e","Type":"ContainerDied","Data":"041d79e705ecfb23eda5f9834e90a8d0f8cb9a3afaf6887d428268059e1028af"}
Sep 18 19:00:02.491530 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.491495    2568 generic.go:356] "Generic (PLEG): container finished" podID="cc1be9f1-6611-4421-a5ab-96dbc0d13bee" containerID="59610dea0d40b97c62debfa432df742e4460837b40f2027cff7558e246f3b4df" exitCode=0
Sep 18 19:00:02.491594 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.491551    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn" event={"ID":"cc1be9f1-6611-4421-a5ab-96dbc0d13bee","Type":"ContainerDied","Data":"59610dea0d40b97c62debfa432df742e4460837b40f2027cff7558e246f3b4df"}
Sep 18 19:00:02.493039 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.493020    2568 generic.go:356] "Generic (PLEG): container finished" podID="73464364-b803-4183-a5fa-d2e0b68e4b64" containerID="15fa15a049a8163c28384871f315a9c3980a70262be5cb79cec4291f57b18546" exitCode=0
Sep 18 19:00:02.493162 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.493096    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h" event={"ID":"73464364-b803-4183-a5fa-d2e0b68e4b64","Type":"ContainerDied","Data":"15fa15a049a8163c28384871f315a9c3980a70262be5cb79cec4291f57b18546"}
Sep 18 19:00:02.494470 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.494441    2568 generic.go:356] "Generic (PLEG): container finished" podID="a5a53f7b-9b25-4866-94d5-3316689d7d73" containerID="37842cc2b671f31972b6d11b4ede9898f7b6b730516549588f07dc5662d26166" exitCode=0
Sep 18 19:00:02.494557 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.494499    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh" event={"ID":"a5a53f7b-9b25-4866-94d5-3316689d7d73","Type":"ContainerDied","Data":"37842cc2b671f31972b6d11b4ede9898f7b6b730516549588f07dc5662d26166"}
Sep 18 19:00:02.527182 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:02.527108    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h" podStartSLOduration=2.52708841 podStartE2EDuration="2.52708841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:01.721591928 +0000 UTC m=+931.159852132" watchObservedRunningTime="2026-09-18 19:00:02.52708841 +0000 UTC m=+931.965348614"
Sep 18 19:00:03.692301 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:03.692274    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"
Sep 18 19:00:03.765940 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:03.765871    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"
Sep 18 19:00:03.804530 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:03.804497    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"
Sep 18 19:00:03.807626 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:03.807606    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"
Sep 18 19:00:04.507086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.507059    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"
Sep 18 19:00:04.507301 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.507101    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh" event={"ID":"a5a53f7b-9b25-4866-94d5-3316689d7d73","Type":"ContainerDied","Data":"52364fba4abc7ef5f06097afdbeb8ef0abdd753ca846453859a68060a7d2e3e8"}
Sep 18 19:00:04.507301 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.507163    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52364fba4abc7ef5f06097afdbeb8ef0abdd753ca846453859a68060a7d2e3e8"
Sep 18 19:00:04.508608 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.508589    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"
Sep 18 19:00:04.508718 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.508609    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42" event={"ID":"63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e","Type":"ContainerDied","Data":"43912f8a6540463b8f528b5e539223be3d254f921861540e3b415be9904a4553"}
Sep 18 19:00:04.508718 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.508645    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43912f8a6540463b8f528b5e539223be3d254f921861540e3b415be9904a4553"
Sep 18 19:00:04.510073 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.510047    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn" event={"ID":"cc1be9f1-6611-4421-a5ab-96dbc0d13bee","Type":"ContainerDied","Data":"0bd71d327a583421e55f9f9575d4382bf4854aa53df6e277494b721f8407530a"}
Sep 18 19:00:04.510165 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.510091    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bd71d327a583421e55f9f9575d4382bf4854aa53df6e277494b721f8407530a"
Sep 18 19:00:04.510165 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.510141    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"
Sep 18 19:00:04.511872 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.511856    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"
Sep 18 19:00:04.511962 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.511872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h" event={"ID":"73464364-b803-4183-a5fa-d2e0b68e4b64","Type":"ContainerDied","Data":"ecff38f3a7b63e2d3c89f2e678810f080197f2128b21aa512a048c17d6cbc4bb"}
Sep 18 19:00:04.511962 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:04.511913    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecff38f3a7b63e2d3c89f2e678810f080197f2128b21aa512a048c17d6cbc4bb"
Sep 18 19:00:10.548164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:10.548077    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerID="117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e" exitCode=0
Sep 18 19:00:10.548164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:10.548145    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" event={"ID":"2c220890-39c8-4869-9c25-3b9027a7b3ee","Type":"ContainerDied","Data":"117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"}
Sep 18 19:00:10.548717 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:10.548616    2568 scope.go:117] "RemoveContainer" containerID="117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"
Sep 18 19:00:10.549836 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:10.549810    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:00:10.549894 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:10.549875    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:00:10.549936 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:10.549907    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:00:11.556581 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:11.556536    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" event={"ID":"2c220890-39c8-4869-9c25-3b9027a7b3ee","Type":"ContainerStarted","Data":"03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f"}
Sep 18 19:00:12.098909 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:12.098875    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 19:00:12.099134 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:12.099111    2568 patch_prober.go:28] interesting pod/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 19:00:12.099227 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:12.099205    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 19:00:13.098535 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:13.098482    2568 patch_prober.go:28] interesting pod/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 19:00:13.098917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:13.098613    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 19:00:14.098161 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:14.098122    2568 patch_prober.go:28] interesting pod/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 19:00:14.098332 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:14.098177    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 19:00:15.099011 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:15.098979    2568 patch_prober.go:28] interesting pod/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 19:00:15.099427 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:15.099031    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 19:00:16.099303 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:16.099260    2568 patch_prober.go:28] interesting pod/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 18 19:00:16.099738 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:16.099348    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 18 19:00:17.102752 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:17.102722    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 19:00:17.103264 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:17.103101    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 19:00:17.104635 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:17.104615    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 19:00:25.488831 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.488796    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"]
Sep 18 19:00:25.489306 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489292    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e" containerName="cleanup"
Sep 18 19:00:25.489306 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489307    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e" containerName="cleanup"
Sep 18 19:00:25.489382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489327    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73464364-b803-4183-a5fa-d2e0b68e4b64" containerName="cleanup"
Sep 18 19:00:25.489382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489333    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="73464364-b803-4183-a5fa-d2e0b68e4b64" containerName="cleanup"
Sep 18 19:00:25.489382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489341    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5a53f7b-9b25-4866-94d5-3316689d7d73" containerName="cleanup"
Sep 18 19:00:25.489382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489346    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5a53f7b-9b25-4866-94d5-3316689d7d73" containerName="cleanup"
Sep 18 19:00:25.489382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489362    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc1be9f1-6611-4421-a5ab-96dbc0d13bee" containerName="cleanup"
Sep 18 19:00:25.489382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489370    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1be9f1-6611-4421-a5ab-96dbc0d13bee" containerName="cleanup"
Sep 18 19:00:25.489600 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489440    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5a53f7b-9b25-4866-94d5-3316689d7d73" containerName="cleanup"
Sep 18 19:00:25.489600 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489455    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc1be9f1-6611-4421-a5ab-96dbc0d13bee" containerName="cleanup"
Sep 18 19:00:25.489600 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489462    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e" containerName="cleanup"
Sep 18 19:00:25.489600 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.489467    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="73464364-b803-4183-a5fa-d2e0b68e4b64" containerName="cleanup"
Sep 18 19:00:25.491688 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.491672    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:25.494541 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.494521    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:00:25.494811 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.494791    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:00:25.502281 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.502260    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"]
Sep 18 19:00:25.593161 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.593126    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72d2414-13c8-49de-97cd-07c5be5c36ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx\" (UID: \"b72d2414-13c8-49de-97cd-07c5be5c36ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:25.694191 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.694158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72d2414-13c8-49de-97cd-07c5be5c36ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx\" (UID: \"b72d2414-13c8-49de-97cd-07c5be5c36ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:25.694833 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.694815    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72d2414-13c8-49de-97cd-07c5be5c36ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx\" (UID: \"b72d2414-13c8-49de-97cd-07c5be5c36ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:25.803442 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:25.803352    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:26.094647 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:26.094620    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"]
Sep 18 19:00:26.096867 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:00:26.096843    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72d2414_13c8_49de_97cd_07c5be5c36ae.slice/crio-01d860dd5b83710fde4e91cffa7e4074a7e4b3c1676151a5d4bd8f8549afe308 WatchSource:0}: Error finding container 01d860dd5b83710fde4e91cffa7e4074a7e4b3c1676151a5d4bd8f8549afe308: Status 404 returned error can't find the container with id 01d860dd5b83710fde4e91cffa7e4074a7e4b3c1676151a5d4bd8f8549afe308
Sep 18 19:00:26.641346 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:26.641306    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx" event={"ID":"b72d2414-13c8-49de-97cd-07c5be5c36ae","Type":"ContainerStarted","Data":"e788d7aa2ed3f07cd0cc105f05538d0e632f4683ad593d580973d4fb2d5e08a2"}
Sep 18 19:00:26.641722 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:26.641391    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx" event={"ID":"b72d2414-13c8-49de-97cd-07c5be5c36ae","Type":"ContainerStarted","Data":"01d860dd5b83710fde4e91cffa7e4074a7e4b3c1676151a5d4bd8f8549afe308"}
Sep 18 19:00:26.686237 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:26.686139    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx" podStartSLOduration=1.686123364 podStartE2EDuration="1.686123364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:00:26.68338544 +0000 UTC m=+956.121645644" watchObservedRunningTime="2026-09-18 19:00:26.686123364 +0000 UTC m=+956.124383568"
Sep 18 19:00:27.648815 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:27.648778    2568 generic.go:356] "Generic (PLEG): container finished" podID="b72d2414-13c8-49de-97cd-07c5be5c36ae" containerID="e788d7aa2ed3f07cd0cc105f05538d0e632f4683ad593d580973d4fb2d5e08a2" exitCode=0
Sep 18 19:00:27.649197 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:27.648861    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx" event={"ID":"b72d2414-13c8-49de-97cd-07c5be5c36ae","Type":"ContainerDied","Data":"e788d7aa2ed3f07cd0cc105f05538d0e632f4683ad593d580973d4fb2d5e08a2"}
Sep 18 19:00:28.804111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:28.804086    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:28.928192 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:28.928102    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72d2414-13c8-49de-97cd-07c5be5c36ae-maas-api-service-ca\") pod \"b72d2414-13c8-49de-97cd-07c5be5c36ae\" (UID: \"b72d2414-13c8-49de-97cd-07c5be5c36ae\") "
Sep 18 19:00:28.928475 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:28.928449    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72d2414-13c8-49de-97cd-07c5be5c36ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b72d2414-13c8-49de-97cd-07c5be5c36ae" (UID: "b72d2414-13c8-49de-97cd-07c5be5c36ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:00:29.029750 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:29.029718    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72d2414-13c8-49de-97cd-07c5be5c36ae-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:29.662232 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:29.662198    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx" event={"ID":"b72d2414-13c8-49de-97cd-07c5be5c36ae","Type":"ContainerDied","Data":"01d860dd5b83710fde4e91cffa7e4074a7e4b3c1676151a5d4bd8f8549afe308"}
Sep 18 19:00:29.662402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:29.662237    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"
Sep 18 19:00:29.662402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:29.662249    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01d860dd5b83710fde4e91cffa7e4074a7e4b3c1676151a5d4bd8f8549afe308"
Sep 18 19:00:36.428233 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:36.428200    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"]
Sep 18 19:00:36.439016 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:36.438987    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4465858-b9gnx"]
Sep 18 19:00:36.696256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:36.696168    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"]
Sep 18 19:00:36.696440 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:36.696399    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy" containerID="cri-o://03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f" gracePeriod=30
Sep 18 19:00:37.055360 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:37.055325    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72d2414-13c8-49de-97cd-07c5be5c36ae" path="/var/lib/kubelet/pods/b72d2414-13c8-49de-97cd-07c5be5c36ae/volumes"
Sep 18 19:00:41.959731 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:41.959667    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 19:00:42.059756 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059723    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-data\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.059951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059766    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r97g\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-kube-api-access-5r97g\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.059951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059784    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c220890-39c8-4869-9c25-3b9027a7b3ee-istiod-ca-cert\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.059951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059799    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-certs\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.059951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059821    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-socket\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.059951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059887    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-token\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.059951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059935    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-credential-socket\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.060285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.059964    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-podinfo\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.060285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060043    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:42.060285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060083    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-envoy\") pod \"2c220890-39c8-4869-9c25-3b9027a7b3ee\" (UID: \"2c220890-39c8-4869-9c25-3b9027a7b3ee\") "
Sep 18 19:00:42.060285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060115    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:42.060285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060138    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:42.060567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060334    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:42.060567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060421    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c220890-39c8-4869-9c25-3b9027a7b3ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:00:42.060687 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060655    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c220890-39c8-4869-9c25-3b9027a7b3ee-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.060687 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060676    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.060788 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060689    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.060788 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060702    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.060788 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.060717    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.062356 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.062327    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:00:42.062463 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.062353    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:00:42.062463 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.062390    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:42.062463 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.062409    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-kube-api-access-5r97g" (OuterVolumeSpecName: "kube-api-access-5r97g") pod "2c220890-39c8-4869-9c25-3b9027a7b3ee" (UID: "2c220890-39c8-4869-9c25-3b9027a7b3ee"). InnerVolumeSpecName "kube-api-access-5r97g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:00:42.161373 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.161339    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r97g\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-kube-api-access-5r97g\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.161373 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.161372    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.161601 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.161384    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.161601 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.161395    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c220890-39c8-4869-9c25-3b9027a7b3ee-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:00:42.749430 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.749395    2568 generic.go:356] "Generic (PLEG): container finished" podID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerID="03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f" exitCode=0
Sep 18 19:00:42.749614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.749444    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"
Sep 18 19:00:42.749614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.749454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" event={"ID":"2c220890-39c8-4869-9c25-3b9027a7b3ee","Type":"ContainerDied","Data":"03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f"}
Sep 18 19:00:42.749614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.749499    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92" event={"ID":"2c220890-39c8-4869-9c25-3b9027a7b3ee","Type":"ContainerDied","Data":"07dee786477cc1ff63b099f67a0ee4731988061cfa7a58b31de92c3ed20371ec"}
Sep 18 19:00:42.749614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.749554    2568 scope.go:117] "RemoveContainer" containerID="03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f"
Sep 18 19:00:42.760804 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.760787    2568 scope.go:117] "RemoveContainer" containerID="117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"
Sep 18 19:00:42.769325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.769304    2568 scope.go:117] "RemoveContainer" containerID="03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f"
Sep 18 19:00:42.769633 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:00:42.769611    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f\": container with ID starting with 03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f not found: ID does not exist" containerID="03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f"
Sep 18 19:00:42.769694 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.769641    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f"} err="failed to get container status \"03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f\": rpc error: code = NotFound desc = could not find container \"03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f\": container with ID starting with 03ff072a4549520af6c6377c9e25c8e082f2e632ac1f6faa98f924591f97220f not found: ID does not exist"
Sep 18 19:00:42.769694 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.769657    2568 scope.go:117] "RemoveContainer" containerID="117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"
Sep 18 19:00:42.769927 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:00:42.769909    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e\": container with ID starting with 117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e not found: ID does not exist" containerID="117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"
Sep 18 19:00:42.769976 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.769933    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e"} err="failed to get container status \"117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e\": rpc error: code = NotFound desc = could not find container \"117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e\": container with ID starting with 117af89a7e549734f6b1b4dd67798f80e8c0caba787002a31dead73471e85d7e not found: ID does not exist"
Sep 18 19:00:42.787456 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.787431    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"]
Sep 18 19:00:42.802863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:42.802838    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f4465858-gw-openshift-default-67c7b4b4-78j92"]
Sep 18 19:00:43.055347 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:00:43.055249    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" path="/var/lib/kubelet/pods/2c220890-39c8-4869-9c25-3b9027a7b3ee/volumes"
Sep 18 19:01:04.270969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.270883    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"]
Sep 18 19:01:04.271446 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271388    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy"
Sep 18 19:01:04.271446 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271403    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy"
Sep 18 19:01:04.271446 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271425    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72d2414-13c8-49de-97cd-07c5be5c36ae" containerName="revoke-keys"
Sep 18 19:01:04.271446 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271430    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72d2414-13c8-49de-97cd-07c5be5c36ae" containerName="revoke-keys"
Sep 18 19:01:04.271609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271506    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72d2414-13c8-49de-97cd-07c5be5c36ae" containerName="revoke-keys"
Sep 18 19:01:04.271609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271529    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy"
Sep 18 19:01:04.271609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.271541    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy"
Sep 18 19:01:04.274769 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.274752    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:04.281933 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.281912    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:01:04.282660 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.282641    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:01:04.318238 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.318207    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"]
Sep 18 19:01:04.394331 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.394289    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a0ff752-97f2-47ae-b836-6751e8ca77ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-fb8843-g6kxd\" (UID: \"4a0ff752-97f2-47ae-b836-6751e8ca77ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:04.495793 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.495755    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a0ff752-97f2-47ae-b836-6751e8ca77ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-fb8843-g6kxd\" (UID: \"4a0ff752-97f2-47ae-b836-6751e8ca77ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:04.496392 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.496373    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a0ff752-97f2-47ae-b836-6751e8ca77ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-fb8843-g6kxd\" (UID: \"4a0ff752-97f2-47ae-b836-6751e8ca77ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:04.584999 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.584900    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:04.927529 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:04.927491    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"]
Sep 18 19:01:04.929567 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:01:04.929542    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0ff752_97f2_47ae_b836_6751e8ca77ab.slice/crio-8f3ebaa8e5e714970dfe3d24d0135b1f68a1a4db8dda369fbcce7566dc263f04 WatchSource:0}: Error finding container 8f3ebaa8e5e714970dfe3d24d0135b1f68a1a4db8dda369fbcce7566dc263f04: Status 404 returned error can't find the container with id 8f3ebaa8e5e714970dfe3d24d0135b1f68a1a4db8dda369fbcce7566dc263f04
Sep 18 19:01:05.883322 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:05.883221    2568 generic.go:356] "Generic (PLEG): container finished" podID="4a0ff752-97f2-47ae-b836-6751e8ca77ab" containerID="04d2de4c4fa8b3b047da95a46a111fcf0cfe6a03c0cafcc6bd6304fbf1b128ce" exitCode=0
Sep 18 19:01:05.883322 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:05.883303    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd" event={"ID":"4a0ff752-97f2-47ae-b836-6751e8ca77ab","Type":"ContainerDied","Data":"04d2de4c4fa8b3b047da95a46a111fcf0cfe6a03c0cafcc6bd6304fbf1b128ce"}
Sep 18 19:01:05.883803 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:05.883354    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd" event={"ID":"4a0ff752-97f2-47ae-b836-6751e8ca77ab","Type":"ContainerStarted","Data":"8f3ebaa8e5e714970dfe3d24d0135b1f68a1a4db8dda369fbcce7566dc263f04"}
Sep 18 19:01:07.040140 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.040110    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:07.225378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.225341    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a0ff752-97f2-47ae-b836-6751e8ca77ab-maas-api-service-ca\") pod \"4a0ff752-97f2-47ae-b836-6751e8ca77ab\" (UID: \"4a0ff752-97f2-47ae-b836-6751e8ca77ab\") "
Sep 18 19:01:07.225805 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.225780    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a0ff752-97f2-47ae-b836-6751e8ca77ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a0ff752-97f2-47ae-b836-6751e8ca77ab" (UID: "4a0ff752-97f2-47ae-b836-6751e8ca77ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:07.326610 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.326572    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a0ff752-97f2-47ae-b836-6751e8ca77ab-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:07.895875 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.895847    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"
Sep 18 19:01:07.896039 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.895847    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd" event={"ID":"4a0ff752-97f2-47ae-b836-6751e8ca77ab","Type":"ContainerDied","Data":"8f3ebaa8e5e714970dfe3d24d0135b1f68a1a4db8dda369fbcce7566dc263f04"}
Sep 18 19:01:07.896039 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:07.895967    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f3ebaa8e5e714970dfe3d24d0135b1f68a1a4db8dda369fbcce7566dc263f04"
Sep 18 19:01:16.732447 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.732409    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"]
Sep 18 19:01:16.732885 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.732872    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy"
Sep 18 19:01:16.732929 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.732886    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c220890-39c8-4869-9c25-3b9027a7b3ee" containerName="istio-proxy"
Sep 18 19:01:16.732929 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.732904    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0ff752-97f2-47ae-b836-6751e8ca77ab" containerName="revoke-keys"
Sep 18 19:01:16.732929 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.732910    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0ff752-97f2-47ae-b836-6751e8ca77ab" containerName="revoke-keys"
Sep 18 19:01:16.733022 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.732975    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0ff752-97f2-47ae-b836-6751e8ca77ab" containerName="revoke-keys"
Sep 18 19:01:16.736266 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.736250    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:16.739657 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.739625    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:01:16.739657 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.739628    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:01:16.747424 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.747397    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"]
Sep 18 19:01:16.821282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.821237    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be98e68-f318-4a2d-8f63-aab58da86acc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2\" (UID: \"7be98e68-f318-4a2d-8f63-aab58da86acc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:16.922256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.922220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be98e68-f318-4a2d-8f63-aab58da86acc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2\" (UID: \"7be98e68-f318-4a2d-8f63-aab58da86acc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:16.922875 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:16.922855    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be98e68-f318-4a2d-8f63-aab58da86acc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2\" (UID: \"7be98e68-f318-4a2d-8f63-aab58da86acc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:17.046995 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:17.046911    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:17.324253 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:17.324225    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"]
Sep 18 19:01:17.325369 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:01:17.325341    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be98e68_f318_4a2d_8f63_aab58da86acc.slice/crio-de717ce5075158f6fb75aab1bfc3fd8194267e91b23f5a2c19ac65d09c38dc35 WatchSource:0}: Error finding container de717ce5075158f6fb75aab1bfc3fd8194267e91b23f5a2c19ac65d09c38dc35: Status 404 returned error can't find the container with id de717ce5075158f6fb75aab1bfc3fd8194267e91b23f5a2c19ac65d09c38dc35
Sep 18 19:01:17.949011 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:17.948925    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2" event={"ID":"7be98e68-f318-4a2d-8f63-aab58da86acc","Type":"ContainerStarted","Data":"6ee4612ff41bbb9562ae5ee6a9d97678dacb1b932610266760c990e4a6ad30f2"}
Sep 18 19:01:17.949011 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:17.948980    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2" event={"ID":"7be98e68-f318-4a2d-8f63-aab58da86acc","Type":"ContainerStarted","Data":"de717ce5075158f6fb75aab1bfc3fd8194267e91b23f5a2c19ac65d09c38dc35"}
Sep 18 19:01:17.971096 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:17.971047    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2" podStartSLOduration=1.971034355 podStartE2EDuration="1.971034355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:01:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:01:17.968121763 +0000 UTC m=+1007.406381958" watchObservedRunningTime="2026-09-18 19:01:17.971034355 +0000 UTC m=+1007.409294558"
Sep 18 19:01:18.956418 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:18.956387    2568 generic.go:356] "Generic (PLEG): container finished" podID="7be98e68-f318-4a2d-8f63-aab58da86acc" containerID="6ee4612ff41bbb9562ae5ee6a9d97678dacb1b932610266760c990e4a6ad30f2" exitCode=0
Sep 18 19:01:18.956793 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:18.956479    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2" event={"ID":"7be98e68-f318-4a2d-8f63-aab58da86acc","Type":"ContainerDied","Data":"6ee4612ff41bbb9562ae5ee6a9d97678dacb1b932610266760c990e4a6ad30f2"}
Sep 18 19:01:20.103746 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.103715    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:20.157213 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.157182    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be98e68-f318-4a2d-8f63-aab58da86acc-maas-api-service-ca\") pod \"7be98e68-f318-4a2d-8f63-aab58da86acc\" (UID: \"7be98e68-f318-4a2d-8f63-aab58da86acc\") "
Sep 18 19:01:20.157925 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.157557    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be98e68-f318-4a2d-8f63-aab58da86acc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7be98e68-f318-4a2d-8f63-aab58da86acc" (UID: "7be98e68-f318-4a2d-8f63-aab58da86acc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:20.157925 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.157772    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be98e68-f318-4a2d-8f63-aab58da86acc-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.392350 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.392316    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"]
Sep 18 19:01:20.408126 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.408098    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-fb8843-29829300-lpp42"]
Sep 18 19:01:20.453001 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.452967    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"]
Sep 18 19:01:20.453225 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.453186    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq" podUID="90d57164-6148-4db1-8c41-84635e0a2c31" containerName="payload-processing" containerID="cri-o://b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb" gracePeriod=30
Sep 18 19:01:20.464475 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.464443    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"]
Sep 18 19:01:20.464725 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.464699    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk" podUID="0571f833-7e15-420b-8dbf-221454fd4aa0" containerName="payload-pre-processing" containerID="cri-o://93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd" gracePeriod=30
Sep 18 19:01:20.756892 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.756859    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 19:01:20.759014 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.758988    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 19:01:20.864722 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.864690    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vfps\" (UniqueName: \"kubernetes.io/projected/90d57164-6148-4db1-8c41-84635e0a2c31-kube-api-access-4vfps\") pod \"90d57164-6148-4db1-8c41-84635e0a2c31\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") "
Sep 18 19:01:20.864887 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.864734    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0571f833-7e15-420b-8dbf-221454fd4aa0-plugins-config-volume\") pod \"0571f833-7e15-420b-8dbf-221454fd4aa0\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") "
Sep 18 19:01:20.864887 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.864865    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxw7q\" (UniqueName: \"kubernetes.io/projected/0571f833-7e15-420b-8dbf-221454fd4aa0-kube-api-access-xxw7q\") pod \"0571f833-7e15-420b-8dbf-221454fd4aa0\" (UID: \"0571f833-7e15-420b-8dbf-221454fd4aa0\") "
Sep 18 19:01:20.865025 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.864962    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90d57164-6148-4db1-8c41-84635e0a2c31-plugins-config-volume\") pod \"90d57164-6148-4db1-8c41-84635e0a2c31\" (UID: \"90d57164-6148-4db1-8c41-84635e0a2c31\") "
Sep 18 19:01:20.865129 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.865104    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0571f833-7e15-420b-8dbf-221454fd4aa0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0571f833-7e15-420b-8dbf-221454fd4aa0" (UID: "0571f833-7e15-420b-8dbf-221454fd4aa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:20.865335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.865309    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90d57164-6148-4db1-8c41-84635e0a2c31-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "90d57164-6148-4db1-8c41-84635e0a2c31" (UID: "90d57164-6148-4db1-8c41-84635e0a2c31"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:20.865435 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.865318    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0571f833-7e15-420b-8dbf-221454fd4aa0-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.867452 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.867429    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90d57164-6148-4db1-8c41-84635e0a2c31-kube-api-access-4vfps" (OuterVolumeSpecName: "kube-api-access-4vfps") pod "90d57164-6148-4db1-8c41-84635e0a2c31" (UID: "90d57164-6148-4db1-8c41-84635e0a2c31"). InnerVolumeSpecName "kube-api-access-4vfps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:20.867452 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.867442    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0571f833-7e15-420b-8dbf-221454fd4aa0-kube-api-access-xxw7q" (OuterVolumeSpecName: "kube-api-access-xxw7q") pod "0571f833-7e15-420b-8dbf-221454fd4aa0" (UID: "0571f833-7e15-420b-8dbf-221454fd4aa0"). InnerVolumeSpecName "kube-api-access-xxw7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:01:20.952786 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.952706    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"]
Sep 18 19:01:20.965939 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.965912    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vfps\" (UniqueName: \"kubernetes.io/projected/90d57164-6148-4db1-8c41-84635e0a2c31-kube-api-access-4vfps\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.965939 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.965936    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxw7q\" (UniqueName: \"kubernetes.io/projected/0571f833-7e15-420b-8dbf-221454fd4aa0-kube-api-access-xxw7q\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.965939 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.965945    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90d57164-6148-4db1-8c41-84635e0a2c31-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:20.968140 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.968115    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"
Sep 18 19:01:20.968254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.968147    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2" event={"ID":"7be98e68-f318-4a2d-8f63-aab58da86acc","Type":"ContainerDied","Data":"de717ce5075158f6fb75aab1bfc3fd8194267e91b23f5a2c19ac65d09c38dc35"}
Sep 18 19:01:20.968254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.968190    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de717ce5075158f6fb75aab1bfc3fd8194267e91b23f5a2c19ac65d09c38dc35"
Sep 18 19:01:20.969499 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.969476    2568 generic.go:356] "Generic (PLEG): container finished" podID="90d57164-6148-4db1-8c41-84635e0a2c31" containerID="b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb" exitCode=0
Sep 18 19:01:20.969608 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.969540    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"
Sep 18 19:01:20.969608 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.969563    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq" event={"ID":"90d57164-6148-4db1-8c41-84635e0a2c31","Type":"ContainerDied","Data":"b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb"}
Sep 18 19:01:20.969734 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.969617    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq" event={"ID":"90d57164-6148-4db1-8c41-84635e0a2c31","Type":"ContainerDied","Data":"be4dab378f5b28c1bf078f568ddfa5eaa959b56e5cc18cddf59cfd1b294abd87"}
Sep 18 19:01:20.969734 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.969650    2568 scope.go:117] "RemoveContainer" containerID="b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb"
Sep 18 19:01:20.971085 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.970858    2568 generic.go:356] "Generic (PLEG): container finished" podID="0571f833-7e15-420b-8dbf-221454fd4aa0" containerID="93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd" exitCode=0
Sep 18 19:01:20.971085 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.970888    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk" event={"ID":"0571f833-7e15-420b-8dbf-221454fd4aa0","Type":"ContainerDied","Data":"93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd"}
Sep 18 19:01:20.971085 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.970914    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"
Sep 18 19:01:20.971085 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.970925    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk" event={"ID":"0571f833-7e15-420b-8dbf-221454fd4aa0","Type":"ContainerDied","Data":"401d38ed01035e890047ce3dd490048a3a89be47e39e991104f365551c43be7d"}
Sep 18 19:01:20.976912 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.976888    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-fb8843-g6kxd"]
Sep 18 19:01:20.980784 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.980770    2568 scope.go:117] "RemoveContainer" containerID="b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb"
Sep 18 19:01:20.981081 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:01:20.981055    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb\": container with ID starting with b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb not found: ID does not exist" containerID="b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb"
Sep 18 19:01:20.981147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.981093    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb"} err="failed to get container status \"b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb\": rpc error: code = NotFound desc = could not find container \"b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb\": container with ID starting with b6803d6b17ee9d0beac8c9f176abc2c509985fd14517d5a22b923627f9cd00bb not found: ID does not exist"
Sep 18 19:01:20.981147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.981120    2568 scope.go:117] "RemoveContainer" containerID="93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd"
Sep 18 19:01:20.989452 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.989432    2568 scope.go:117] "RemoveContainer" containerID="93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd"
Sep 18 19:01:20.989798 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:01:20.989776    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd\": container with ID starting with 93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd not found: ID does not exist" containerID="93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd"
Sep 18 19:01:20.989884 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:20.989803    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd"} err="failed to get container status \"93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd\": rpc error: code = NotFound desc = could not find container \"93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd\": container with ID starting with 93e6388bc4e23e2f31142d462811807800cff35a44d2f1cc5f85291324695ecd not found: ID does not exist"
Sep 18 19:01:21.035732 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.035701    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"]
Sep 18 19:01:21.046003 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.045976    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-fb8843-5df889c6b6-9jpnk"]
Sep 18 19:01:21.055075 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.055045    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0571f833-7e15-420b-8dbf-221454fd4aa0" path="/var/lib/kubelet/pods/0571f833-7e15-420b-8dbf-221454fd4aa0/volumes"
Sep 18 19:01:21.055493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.055474    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a0ff752-97f2-47ae-b836-6751e8ca77ab" path="/var/lib/kubelet/pods/4a0ff752-97f2-47ae-b836-6751e8ca77ab/volumes"
Sep 18 19:01:21.055834 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.055816    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e" path="/var/lib/kubelet/pods/63996dd5-a52f-4d8c-9e37-ef1c2e4f2d4e/volumes"
Sep 18 19:01:21.082059 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.082025    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"]
Sep 18 19:01:21.092389 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:21.092364    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-fb8843-75b6d98bf5-2t7cq"]
Sep 18 19:01:23.054451 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:23.054414    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90d57164-6148-4db1-8c41-84635e0a2c31" path="/var/lib/kubelet/pods/90d57164-6148-4db1-8c41-84635e0a2c31/volumes"
Sep 18 19:01:29.793889 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:29.793785    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"]
Sep 18 19:01:29.807274 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:29.807227    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-980e4a32-jv6f2"]
Sep 18 19:01:31.058564 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:31.058535    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be98e68-f318-4a2d-8f63-aab58da86acc" path="/var/lib/kubelet/pods/7be98e68-f318-4a2d-8f63-aab58da86acc/volumes"
Sep 18 19:01:32.403638 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.403599    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"]
Sep 18 19:01:32.404231 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404210    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90d57164-6148-4db1-8c41-84635e0a2c31" containerName="payload-processing"
Sep 18 19:01:32.404312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404234    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="90d57164-6148-4db1-8c41-84635e0a2c31" containerName="payload-processing"
Sep 18 19:01:32.404312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404253    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0571f833-7e15-420b-8dbf-221454fd4aa0" containerName="payload-pre-processing"
Sep 18 19:01:32.404312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404262    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0571f833-7e15-420b-8dbf-221454fd4aa0" containerName="payload-pre-processing"
Sep 18 19:01:32.404312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404270    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be98e68-f318-4a2d-8f63-aab58da86acc" containerName="revoke-keys"
Sep 18 19:01:32.404312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404279    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be98e68-f318-4a2d-8f63-aab58da86acc" containerName="revoke-keys"
Sep 18 19:01:32.404592 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404379    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be98e68-f318-4a2d-8f63-aab58da86acc" containerName="revoke-keys"
Sep 18 19:01:32.404592 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404393    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0571f833-7e15-420b-8dbf-221454fd4aa0" containerName="payload-pre-processing"
Sep 18 19:01:32.404592 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.404405    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="90d57164-6148-4db1-8c41-84635e0a2c31" containerName="payload-processing"
Sep 18 19:01:32.408955 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.408936    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.418603 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.418573    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3341b20d-openshift-default-dockercfg-cgql7\""
Sep 18 19:01:32.454900 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.454860    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"]
Sep 18 19:01:32.482989 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.482958    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-socket\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483156 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-podinfo\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483156 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483088    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-data\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483156 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-certs\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483158    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-credential-socket\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istiod-ca-cert\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483251    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-envoy\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483365 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483270    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-token\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.483397 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.483373    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5tt2\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-kube-api-access-n5tt2\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584482 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584452    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-data\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584703 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584491    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-certs\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584703 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-credential-socket\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584703 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584578    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istiod-ca-cert\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584882 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-envoy\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584882 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584749    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-token\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.584882 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584813    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5tt2\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-kube-api-access-n5tt2\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584890    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-data\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584894    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-socket\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-podinfo\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.584980    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-certs\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585233 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.585212    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-credential-socket\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585292 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.585240    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-socket\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.585292 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.585250    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istiod-ca-cert\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.586928 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.586902    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-envoy\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.587274 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.587257    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-podinfo\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.650039 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.650011    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5tt2\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-kube-api-access-n5tt2\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.652857 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.652832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-token\") pod \"e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") " pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:32.723164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:32.723127    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:33.034754 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:01:33.034726    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f1178f_5c2e_47c5_b5e2_00bc1913f819.slice/crio-2e1d38d7d25b3db1622b323bdbafe9680b7bf27668317e5aa8b03f4fd0f634d8 WatchSource:0}: Error finding container 2e1d38d7d25b3db1622b323bdbafe9680b7bf27668317e5aa8b03f4fd0f634d8: Status 404 returned error can't find the container with id 2e1d38d7d25b3db1622b323bdbafe9680b7bf27668317e5aa8b03f4fd0f634d8
Sep 18 19:01:33.035352 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:33.035332    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"]
Sep 18 19:01:33.037214 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:33.037182    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:01:33.037295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:33.037246    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:01:33.037295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:33.037274    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:01:34.045830 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:34.045790    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" event={"ID":"09f1178f-5c2e-47c5-b5e2-00bc1913f819","Type":"ContainerStarted","Data":"7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"}
Sep 18 19:01:34.046224 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:34.045891    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" event={"ID":"09f1178f-5c2e-47c5-b5e2-00bc1913f819","Type":"ContainerStarted","Data":"2e1d38d7d25b3db1622b323bdbafe9680b7bf27668317e5aa8b03f4fd0f634d8"}
Sep 18 19:01:34.108144 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:34.108092    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podStartSLOduration=2.108078576 podStartE2EDuration="2.108078576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:01:34.103975882 +0000 UTC m=+1023.542236086" watchObservedRunningTime="2026-09-18 19:01:34.108078576 +0000 UTC m=+1023.546338779"
Sep 18 19:01:34.724244 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:34.724205    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:34.729920 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:34.729895    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:35.054864 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:35.054776    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:35.054864 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:35.054857    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:01:43.711887 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.711857    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"]
Sep 18 19:01:43.715768 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.715749    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:43.719549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.719525    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-3341b20d-dockercfg-f2lhh\""
Sep 18 19:01:43.719685 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.719579    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3341b20d\""
Sep 18 19:01:43.761395 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.761362    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"]
Sep 18 19:01:43.815290 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.815254    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3f827550-0b10-415d-86f8-79de2f16806d-kube-api-access-sgs57\") pod \"payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:43.815450 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.815393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f827550-0b10-415d-86f8-79de2f16806d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:43.916479 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.916441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3f827550-0b10-415d-86f8-79de2f16806d-kube-api-access-sgs57\") pod \"payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:43.916693 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.916577    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f827550-0b10-415d-86f8-79de2f16806d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:43.917276 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.917256    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f827550-0b10-415d-86f8-79de2f16806d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:43.947416 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:43.947384    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3f827550-0b10-415d-86f8-79de2f16806d-kube-api-access-sgs57\") pod \"payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:44.026837 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:44.026813    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:44.325013 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:44.324926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"]
Sep 18 19:01:44.334294 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:01:44.334260    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f827550_0b10_415d_86f8_79de2f16806d.slice/crio-7f2cd3d029f14dfec550291fa79a6393bdb042e2c6ceab0497ab8ca6afe1274f WatchSource:0}: Error finding container 7f2cd3d029f14dfec550291fa79a6393bdb042e2c6ceab0497ab8ca6afe1274f: Status 404 returned error can't find the container with id 7f2cd3d029f14dfec550291fa79a6393bdb042e2c6ceab0497ab8ca6afe1274f
Sep 18 19:01:45.108200 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:45.108166    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn" event={"ID":"3f827550-0b10-415d-86f8-79de2f16806d","Type":"ContainerStarted","Data":"aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce"}
Sep 18 19:01:45.108595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:45.108219    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn" event={"ID":"3f827550-0b10-415d-86f8-79de2f16806d","Type":"ContainerStarted","Data":"7f2cd3d029f14dfec550291fa79a6393bdb042e2c6ceab0497ab8ca6afe1274f"}
Sep 18 19:01:45.108595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:45.108263    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:45.148029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:45.147974    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn" podStartSLOduration=2.147956603 podStartE2EDuration="2.147956603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:01:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:01:45.14666952 +0000 UTC m=+1034.584929723" watchObservedRunningTime="2026-09-18 19:01:45.147956603 +0000 UTC m=+1034.586216809"
Sep 18 19:01:55.405578 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.405538    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"]
Sep 18 19:01:55.409769 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.409746    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:55.412154 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.412135    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:01:55.412280 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.412135    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:01:55.418246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.418224    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"]
Sep 18 19:01:55.552243 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.552194    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c217212f-10ee-4235-8f85-b8522c0e517e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3341b20d-7gk94\" (UID: \"c217212f-10ee-4235-8f85-b8522c0e517e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:55.653778 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.653740    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c217212f-10ee-4235-8f85-b8522c0e517e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3341b20d-7gk94\" (UID: \"c217212f-10ee-4235-8f85-b8522c0e517e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:55.654367 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.654348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c217212f-10ee-4235-8f85-b8522c0e517e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3341b20d-7gk94\" (UID: \"c217212f-10ee-4235-8f85-b8522c0e517e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:55.720852 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:55.720810    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:56.034416 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:56.034313    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"]
Sep 18 19:01:56.051780 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:01:56.051681    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc217212f_10ee_4235_8f85_b8522c0e517e.slice/crio-89a2445cf20a3c83f9d840dd57216cda70b5556a21856dcfeb41432ec5f7383a WatchSource:0}: Error finding container 89a2445cf20a3c83f9d840dd57216cda70b5556a21856dcfeb41432ec5f7383a: Status 404 returned error can't find the container with id 89a2445cf20a3c83f9d840dd57216cda70b5556a21856dcfeb41432ec5f7383a
Sep 18 19:01:56.116825 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:56.116799    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:01:56.179094 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:56.179060    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94" event={"ID":"c217212f-10ee-4235-8f85-b8522c0e517e","Type":"ContainerStarted","Data":"89a2445cf20a3c83f9d840dd57216cda70b5556a21856dcfeb41432ec5f7383a"}
Sep 18 19:01:57.188029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:57.187942    2568 generic.go:356] "Generic (PLEG): container finished" podID="c217212f-10ee-4235-8f85-b8522c0e517e" containerID="893f57e6be2c2965ff5ce6d8620ad0f7d3db71f7fea3fb36c7e657611dbd754c" exitCode=0
Sep 18 19:01:57.188029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:57.188019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94" event={"ID":"c217212f-10ee-4235-8f85-b8522c0e517e","Type":"ContainerDied","Data":"893f57e6be2c2965ff5ce6d8620ad0f7d3db71f7fea3fb36c7e657611dbd754c"}
Sep 18 19:01:58.348024 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:58.347996    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:58.484131 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:58.484101    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c217212f-10ee-4235-8f85-b8522c0e517e-maas-api-service-ca\") pod \"c217212f-10ee-4235-8f85-b8522c0e517e\" (UID: \"c217212f-10ee-4235-8f85-b8522c0e517e\") "
Sep 18 19:01:58.484473 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:58.484447    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c217212f-10ee-4235-8f85-b8522c0e517e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c217212f-10ee-4235-8f85-b8522c0e517e" (UID: "c217212f-10ee-4235-8f85-b8522c0e517e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:01:58.585614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:58.585568    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c217212f-10ee-4235-8f85-b8522c0e517e-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:01:59.202935 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:59.202909    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"
Sep 18 19:01:59.203100 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:59.202952    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94" event={"ID":"c217212f-10ee-4235-8f85-b8522c0e517e","Type":"ContainerDied","Data":"89a2445cf20a3c83f9d840dd57216cda70b5556a21856dcfeb41432ec5f7383a"}
Sep 18 19:01:59.203100 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:01:59.203005    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89a2445cf20a3c83f9d840dd57216cda70b5556a21856dcfeb41432ec5f7383a"
Sep 18 19:02:00.209586 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:00.209551    2568 generic.go:356] "Generic (PLEG): container finished" podID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerID="7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c" exitCode=0
Sep 18 19:02:00.209994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:00.209633    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" event={"ID":"09f1178f-5c2e-47c5-b5e2-00bc1913f819","Type":"ContainerDied","Data":"7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"}
Sep 18 19:02:00.210123 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:00.210107    2568 scope.go:117] "RemoveContainer" containerID="7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"
Sep 18 19:02:00.211331 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:00.211302    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:02:00.211393 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:00.211368    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:02:00.211433 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:00.211399    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:02:01.216772 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:01.216742    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" event={"ID":"09f1178f-5c2e-47c5-b5e2-00bc1913f819","Type":"ContainerStarted","Data":"cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9"}
Sep 18 19:02:01.726308 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:01.726272    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:02:01.726730 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:01.726705    2568 patch_prober.go:28] interesting pod/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 18 19:02:01.726896 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:01.726874    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 18 19:02:02.724498 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:02.724449    2568 patch_prober.go:28] interesting pod/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 18 19:02:02.725035 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:02.724557    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 18 19:02:03.724258 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:03.724210    2568 patch_prober.go:28] interesting pod/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 18 19:02:03.724474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:03.724285    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 18 19:02:04.724486 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:04.724436    2568 patch_prober.go:28] interesting pod/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 18 19:02:04.725037 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:04.724554    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 18 19:02:05.058127 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:05.058052    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:02:05.724471 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:05.724415    2568 patch_prober.go:28] interesting pod/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 18 19:02:05.724731 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:05.724507    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 18 19:02:06.278973 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.278940    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"]
Sep 18 19:02:06.279224 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.279199    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn" podUID="3f827550-0b10-415d-86f8-79de2f16806d" containerName="payload-pre-processing" containerID="cri-o://aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce" gracePeriod=30
Sep 18 19:02:06.606045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.606023    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:02:06.656881 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.656847    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"]
Sep 18 19:02:06.660415 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.660389    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3341b20d-7gk94"]
Sep 18 19:02:06.680337 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.680315    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3f827550-0b10-415d-86f8-79de2f16806d-kube-api-access-sgs57\") pod \"3f827550-0b10-415d-86f8-79de2f16806d\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") "
Sep 18 19:02:06.680469 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.680399    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f827550-0b10-415d-86f8-79de2f16806d-plugins-config-volume\") pod \"3f827550-0b10-415d-86f8-79de2f16806d\" (UID: \"3f827550-0b10-415d-86f8-79de2f16806d\") "
Sep 18 19:02:06.680778 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.680754    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f827550-0b10-415d-86f8-79de2f16806d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f827550-0b10-415d-86f8-79de2f16806d" (UID: "3f827550-0b10-415d-86f8-79de2f16806d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:06.682335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.682313    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f827550-0b10-415d-86f8-79de2f16806d-kube-api-access-sgs57" (OuterVolumeSpecName: "kube-api-access-sgs57") pod "3f827550-0b10-415d-86f8-79de2f16806d" (UID: "3f827550-0b10-415d-86f8-79de2f16806d"). InnerVolumeSpecName "kube-api-access-sgs57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:06.738546 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.728906    2568 patch_prober.go:28] interesting pod/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 18 19:02:06.738546 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.728970    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 18 19:02:06.781929 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.781895    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgs57\" (UniqueName: \"kubernetes.io/projected/3f827550-0b10-415d-86f8-79de2f16806d-kube-api-access-sgs57\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:06.781929 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.781922    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f827550-0b10-415d-86f8-79de2f16806d-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:06.902581 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.902327    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"]
Sep 18 19:02:06.902581 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:06.902526    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy" containerID="cri-o://cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9" gracePeriod=30
Sep 18 19:02:07.063470 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.063435    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c217212f-10ee-4235-8f85-b8522c0e517e" path="/var/lib/kubelet/pods/c217212f-10ee-4235-8f85-b8522c0e517e/volumes"
Sep 18 19:02:07.268357 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.268318    2568 generic.go:356] "Generic (PLEG): container finished" podID="3f827550-0b10-415d-86f8-79de2f16806d" containerID="aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce" exitCode=0
Sep 18 19:02:07.268553 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.268384    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn" event={"ID":"3f827550-0b10-415d-86f8-79de2f16806d","Type":"ContainerDied","Data":"aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce"}
Sep 18 19:02:07.268553 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.268389    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"
Sep 18 19:02:07.268553 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.268431    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn" event={"ID":"3f827550-0b10-415d-86f8-79de2f16806d","Type":"ContainerDied","Data":"7f2cd3d029f14dfec550291fa79a6393bdb042e2c6ceab0497ab8ca6afe1274f"}
Sep 18 19:02:07.268553 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.268456    2568 scope.go:117] "RemoveContainer" containerID="aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce"
Sep 18 19:02:07.279217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.279196    2568 scope.go:117] "RemoveContainer" containerID="aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce"
Sep 18 19:02:07.279594 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:02:07.279566    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce\": container with ID starting with aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce not found: ID does not exist" containerID="aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce"
Sep 18 19:02:07.279685 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.279601    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce"} err="failed to get container status \"aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce\": rpc error: code = NotFound desc = could not find container \"aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce\": container with ID starting with aaa793d20c04beb7ddb867110973175d3c3731cdfe185d6d5f2d9ac151381dce not found: ID does not exist"
Sep 18 19:02:07.315285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.315252    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"]
Sep 18 19:02:07.334664 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:07.334632    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3341b20d-57d8cfc69d-bbpwn"]
Sep 18 19:02:09.059670 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:09.056885    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f827550-0b10-415d-86f8-79de2f16806d" path="/var/lib/kubelet/pods/3f827550-0b10-415d-86f8-79de2f16806d/volumes"
Sep 18 19:02:12.564276 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.564254    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:02:12.640736 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640657    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istiod-ca-cert\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.640736 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640710    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-credential-socket\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.640917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640742    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5tt2\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-kube-api-access-n5tt2\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.640917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640774    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-data\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.640917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640862    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-token\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.640917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640881    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-socket\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.640917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640911    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-certs\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.641166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640945    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-podinfo\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.641166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.640971    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-envoy\") pod \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\" (UID: \"09f1178f-5c2e-47c5-b5e2-00bc1913f819\") "
Sep 18 19:02:12.641166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641090    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:12.641166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641121    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:02:12.641166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641146    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-data" (OuterVolumeSpecName: "istio-data") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:12.641419 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641182    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:12.641419 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641228    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:12.641500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641458    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.641500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641474    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.641500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641483    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.641500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641492    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.641500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.641499    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.643167 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.643139    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:02:12.643298 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.643212    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:02:12.643550 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.643530    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-token" (OuterVolumeSpecName: "istio-token") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:12.643616 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.643561    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-kube-api-access-n5tt2" (OuterVolumeSpecName: "kube-api-access-n5tt2") pod "09f1178f-5c2e-47c5-b5e2-00bc1913f819" (UID: "09f1178f-5c2e-47c5-b5e2-00bc1913f819"). InnerVolumeSpecName "kube-api-access-n5tt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:02:12.742335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.742299    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.742335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.742329    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.742335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.742338    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09f1178f-5c2e-47c5-b5e2-00bc1913f819-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:12.742335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:12.742346    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5tt2\" (UniqueName: \"kubernetes.io/projected/09f1178f-5c2e-47c5-b5e2-00bc1913f819-kube-api-access-n5tt2\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:02:13.305363 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.305332    2568 generic.go:356] "Generic (PLEG): container finished" podID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerID="cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9" exitCode=0
Sep 18 19:02:13.305576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.305386    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"
Sep 18 19:02:13.305576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.305417    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" event={"ID":"09f1178f-5c2e-47c5-b5e2-00bc1913f819","Type":"ContainerDied","Data":"cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9"}
Sep 18 19:02:13.305576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.305480    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb" event={"ID":"09f1178f-5c2e-47c5-b5e2-00bc1913f819","Type":"ContainerDied","Data":"2e1d38d7d25b3db1622b323bdbafe9680b7bf27668317e5aa8b03f4fd0f634d8"}
Sep 18 19:02:13.305576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.305509    2568 scope.go:117] "RemoveContainer" containerID="cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9"
Sep 18 19:02:13.314923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.314876    2568 scope.go:117] "RemoveContainer" containerID="7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"
Sep 18 19:02:13.323316 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.323300    2568 scope.go:117] "RemoveContainer" containerID="cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9"
Sep 18 19:02:13.323685 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:02:13.323666    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9\": container with ID starting with cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9 not found: ID does not exist" containerID="cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9"
Sep 18 19:02:13.323758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.323695    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9"} err="failed to get container status \"cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9\": rpc error: code = NotFound desc = could not find container \"cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9\": container with ID starting with cb637ff7227ac6ab701602b11e93a4500e72b4bbb4ee2d6e793ab28a12645de9 not found: ID does not exist"
Sep 18 19:02:13.323758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.323713    2568 scope.go:117] "RemoveContainer" containerID="7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"
Sep 18 19:02:13.323940 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:02:13.323924    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c\": container with ID starting with 7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c not found: ID does not exist" containerID="7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"
Sep 18 19:02:13.323982 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.323945    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c"} err="failed to get container status \"7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c\": rpc error: code = NotFound desc = could not find container \"7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c\": container with ID starting with 7008da80fab2711534dc4eff2eb870ffeb96c474a0d599b3f5b98ef2a5e0cd7c not found: ID does not exist"
Sep 18 19:02:13.350764 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.350736    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"]
Sep 18 19:02:13.360773 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:13.360748    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3341b20d-openshift-default-648b5b67c6-qbxbb"]
Sep 18 19:02:14.056276 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.056243    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"]
Sep 18 19:02:14.057045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.056981    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c217212f-10ee-4235-8f85-b8522c0e517e" containerName="revoke-keys"
Sep 18 19:02:14.057045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057008    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c217212f-10ee-4235-8f85-b8522c0e517e" containerName="revoke-keys"
Sep 18 19:02:14.057045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057030    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f827550-0b10-415d-86f8-79de2f16806d" containerName="payload-pre-processing"
Sep 18 19:02:14.057045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057038    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f827550-0b10-415d-86f8-79de2f16806d" containerName="payload-pre-processing"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057070    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057078    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057092    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057101    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057189    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057197    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c217212f-10ee-4235-8f85-b8522c0e517e" containerName="revoke-keys"
Sep 18 19:02:14.057293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.057210    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f827550-0b10-415d-86f8-79de2f16806d" containerName="payload-pre-processing"
Sep 18 19:02:14.060497 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.060478    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.063543 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.063509    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-cc7def\""
Sep 18 19:02:14.063626 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.063601    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-cc7def-dockercfg-r6cgm\""
Sep 18 19:02:14.085053 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.085018    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"]
Sep 18 19:02:14.122290 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.122257    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"]
Sep 18 19:02:14.122787 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.122773    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" containerName="istio-proxy"
Sep 18 19:02:14.124893 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.124877    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.156502 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.156475    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2a858c-912e-4bae-bc25-54636e0782df-plugins-config-volume\") pod \"payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") " pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.156639 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.156596    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8srhl\" (UniqueName: \"kubernetes.io/projected/0f2a858c-912e-4bae-bc25-54636e0782df-kube-api-access-8srhl\") pod \"payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") " pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.156698 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.156638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbp46\" (UniqueName: \"kubernetes.io/projected/6755df83-18eb-452c-9339-ba32115574c5-kube-api-access-qbp46\") pod \"payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.156736 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.156697    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6755df83-18eb-452c-9339-ba32115574c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.163301 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.163261    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"]
Sep 18 19:02:14.257826 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.257798    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8srhl\" (UniqueName: \"kubernetes.io/projected/0f2a858c-912e-4bae-bc25-54636e0782df-kube-api-access-8srhl\") pod \"payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") " pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.257998 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.257849    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbp46\" (UniqueName: \"kubernetes.io/projected/6755df83-18eb-452c-9339-ba32115574c5-kube-api-access-qbp46\") pod \"payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.257998 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.257887    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6755df83-18eb-452c-9339-ba32115574c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.257998 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.257915    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2a858c-912e-4bae-bc25-54636e0782df-plugins-config-volume\") pod \"payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") " pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.258604 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.258583    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6755df83-18eb-452c-9339-ba32115574c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.258604 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.258597    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2a858c-912e-4bae-bc25-54636e0782df-plugins-config-volume\") pod \"payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") " pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.271311 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.271287    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbp46\" (UniqueName: \"kubernetes.io/projected/6755df83-18eb-452c-9339-ba32115574c5-kube-api-access-qbp46\") pod \"payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.271468 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.271445    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8srhl\" (UniqueName: \"kubernetes.io/projected/0f2a858c-912e-4bae-bc25-54636e0782df-kube-api-access-8srhl\") pod \"payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") " pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.370549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.370458    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:14.434331 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.434286    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:14.657611 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.657582    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"]
Sep 18 19:02:14.658462 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:02:14.658437    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2a858c_912e_4bae_bc25_54636e0782df.slice/crio-33e1cdadbede8ecb72d35ffbbc7b0899b33c42891d986f5edf0cda6ad2f9c741 WatchSource:0}: Error finding container 33e1cdadbede8ecb72d35ffbbc7b0899b33c42891d986f5edf0cda6ad2f9c741: Status 404 returned error can't find the container with id 33e1cdadbede8ecb72d35ffbbc7b0899b33c42891d986f5edf0cda6ad2f9c741
Sep 18 19:02:14.725069 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:14.725047    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"]
Sep 18 19:02:14.726767 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:02:14.726740    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6755df83_18eb_452c_9339_ba32115574c5.slice/crio-e498ccbfa636e66bb0c659b1c700e8def92fae7e073e463ef2096980ccc7eb82 WatchSource:0}: Error finding container e498ccbfa636e66bb0c659b1c700e8def92fae7e073e463ef2096980ccc7eb82: Status 404 returned error can't find the container with id e498ccbfa636e66bb0c659b1c700e8def92fae7e073e463ef2096980ccc7eb82
Sep 18 19:02:15.054979 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.054945    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09f1178f-5c2e-47c5-b5e2-00bc1913f819" path="/var/lib/kubelet/pods/09f1178f-5c2e-47c5-b5e2-00bc1913f819/volumes"
Sep 18 19:02:15.319549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.319439    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4" event={"ID":"0f2a858c-912e-4bae-bc25-54636e0782df","Type":"ContainerStarted","Data":"68b7e7033f5e08b76ad293b46e53091fdc39a61e13160758e83bd079983fa7c0"}
Sep 18 19:02:15.319910 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.319615    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:15.319910 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.319668    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4" event={"ID":"0f2a858c-912e-4bae-bc25-54636e0782df","Type":"ContainerStarted","Data":"33e1cdadbede8ecb72d35ffbbc7b0899b33c42891d986f5edf0cda6ad2f9c741"}
Sep 18 19:02:15.321047 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.321029    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775" event={"ID":"6755df83-18eb-452c-9339-ba32115574c5","Type":"ContainerStarted","Data":"5db9d04f1789fc8be98c265a42bb75bfb656648b57a48f578aa55e96c80899e8"}
Sep 18 19:02:15.321103 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.321060    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775" event={"ID":"6755df83-18eb-452c-9339-ba32115574c5","Type":"ContainerStarted","Data":"e498ccbfa636e66bb0c659b1c700e8def92fae7e073e463ef2096980ccc7eb82"}
Sep 18 19:02:15.321161 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.321145    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:02:15.348311 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.348257    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4" podStartSLOduration=1.348242994 podStartE2EDuration="1.348242994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:02:15.3453625 +0000 UTC m=+1064.783622705" watchObservedRunningTime="2026-09-18 19:02:15.348242994 +0000 UTC m=+1064.786503229"
Sep 18 19:02:15.373771 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:15.373720    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775" podStartSLOduration=1.3737057240000001 podStartE2EDuration="1.373705724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:02:15.371404368 +0000 UTC m=+1064.809664612" watchObservedRunningTime="2026-09-18 19:02:15.373705724 +0000 UTC m=+1064.811965928"
Sep 18 19:02:26.329933 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:26.329897    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:02:26.330311 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:02:26.329972    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:03:47.321765 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.321730    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"]
Sep 18 19:03:47.327368 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.327342    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.331002 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.330972    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fa6711c6-openshift-default-dockercfg-gn2t8\""
Sep 18 19:03:47.357591 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.357557    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"]
Sep 18 19:03:47.437414 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437372    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-socket\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437622 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437443    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54vrt\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-kube-api-access-54vrt\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437622 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437491    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-credential-socket\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437622 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437533    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-envoy\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437622 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437608    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-certs\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437840 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437702    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istiod-ca-cert\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437840 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437764    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-podinfo\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437840 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-data\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.437969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.437866    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-token\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-socket\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538589    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54vrt\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-kube-api-access-54vrt\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538789 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-credential-socket\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538789 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-envoy\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538789 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538770    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-certs\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istiod-ca-cert\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538860    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-podinfo\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.538923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538881    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-data\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.539057 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-token\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.539057 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.538995    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-socket\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.539166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.539144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-credential-socket\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.539356 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.539333    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-certs\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.539434 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.539356    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-data\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.539510 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.539491    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istiod-ca-cert\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.541176 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.541154    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-envoy\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.541283 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.541267    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-podinfo\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.558983 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.558951    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-token\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.559083 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.559004    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54vrt\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-kube-api-access-54vrt\") pod \"e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") " pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:47.639126 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:47.639038    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:48.251818 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.251792    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"]
Sep 18 19:03:48.253545 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:03:48.253503    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ef8e4c_71d2_4f14_83c9_b7655a94b2a7.slice/crio-beca53648cba4176cc12b09ae299740d9a679b30f3ef7a7bd39f9fdd88450ee1 WatchSource:0}: Error finding container beca53648cba4176cc12b09ae299740d9a679b30f3ef7a7bd39f9fdd88450ee1: Status 404 returned error can't find the container with id beca53648cba4176cc12b09ae299740d9a679b30f3ef7a7bd39f9fdd88450ee1
Sep 18 19:03:48.255863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.255828    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:03:48.255935 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.255895    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:03:48.255935 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.255923    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:03:48.881965 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.881932    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" event={"ID":"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7","Type":"ContainerStarted","Data":"7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"}
Sep 18 19:03:48.882383 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.881988    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" event={"ID":"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7","Type":"ContainerStarted","Data":"beca53648cba4176cc12b09ae299740d9a679b30f3ef7a7bd39f9fdd88450ee1"}
Sep 18 19:03:48.937568 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:48.937497    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podStartSLOduration=1.937482101 podStartE2EDuration="1.937482101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:03:48.935759176 +0000 UTC m=+1158.374019393" watchObservedRunningTime="2026-09-18 19:03:48.937482101 +0000 UTC m=+1158.375742305"
Sep 18 19:03:49.639999 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:49.639962    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:49.645582 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:49.645554    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:49.888130 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:49.888099    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:03:49.889357 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:03:49.889336    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:02.618235 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.618152    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"]
Sep 18 19:04:02.621677 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.621660    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.624211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.624192    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fa6711c6\""
Sep 18 19:04:02.624285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.624267    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fa6711c6-dockercfg-z7dnl\""
Sep 18 19:04:02.634965 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.634942    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"]
Sep 18 19:04:02.671215 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.671182    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"]
Sep 18 19:04:02.675947 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.675925    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.691336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.691285    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"]
Sep 18 19:04:02.692046 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.691968    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d9d11-f274-47e3-acbc-ac6022077d14-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.692159 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.692132    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47rrx\" (UniqueName: \"kubernetes.io/projected/0c9d9d11-f274-47e3-acbc-ac6022077d14-kube-api-access-47rrx\") pod \"payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.793334 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.793299    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d9d11-f274-47e3-acbc-ac6022077d14-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.793504 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.793426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b64lb\" (UniqueName: \"kubernetes.io/projected/6a916c35-a3ac-4104-b559-5d3bb2ff160a-kube-api-access-b64lb\") pod \"payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") " pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.793504 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.793447    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47rrx\" (UniqueName: \"kubernetes.io/projected/0c9d9d11-f274-47e3-acbc-ac6022077d14-kube-api-access-47rrx\") pod \"payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.793504 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.793478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a916c35-a3ac-4104-b559-5d3bb2ff160a-plugins-config-volume\") pod \"payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") " pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.794051 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.794033    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d9d11-f274-47e3-acbc-ac6022077d14-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.802804 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.802776    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47rrx\" (UniqueName: \"kubernetes.io/projected/0c9d9d11-f274-47e3-acbc-ac6022077d14-kube-api-access-47rrx\") pod \"payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:02.894391 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.894314    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b64lb\" (UniqueName: \"kubernetes.io/projected/6a916c35-a3ac-4104-b559-5d3bb2ff160a-kube-api-access-b64lb\") pod \"payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") " pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.894391 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.894359    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a916c35-a3ac-4104-b559-5d3bb2ff160a-plugins-config-volume\") pod \"payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") " pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.895175 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.895157    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a916c35-a3ac-4104-b559-5d3bb2ff160a-plugins-config-volume\") pod \"payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") " pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.906709 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.906681    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b64lb\" (UniqueName: \"kubernetes.io/projected/6a916c35-a3ac-4104-b559-5d3bb2ff160a-kube-api-access-b64lb\") pod \"payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") " pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:02.932425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:02.932399    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:03.007550 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.007510    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:03.221034 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.221011    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"]
Sep 18 19:04:03.222749 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:04:03.222721    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9d9d11_f274_47e3_acbc_ac6022077d14.slice/crio-d2b1d5599fe8bede337a67a086889f75b02eade816c99dec4d07a1e33482f0c1 WatchSource:0}: Error finding container d2b1d5599fe8bede337a67a086889f75b02eade816c99dec4d07a1e33482f0c1: Status 404 returned error can't find the container with id d2b1d5599fe8bede337a67a086889f75b02eade816c99dec4d07a1e33482f0c1
Sep 18 19:04:03.298287 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:04:03.298252    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a916c35_a3ac_4104_b559_5d3bb2ff160a.slice/crio-56e4fde5944b41aeed25e010fc97d2319d03be5375d8e9cd969ecb49e810311f WatchSource:0}: Error finding container 56e4fde5944b41aeed25e010fc97d2319d03be5375d8e9cd969ecb49e810311f: Status 404 returned error can't find the container with id 56e4fde5944b41aeed25e010fc97d2319d03be5375d8e9cd969ecb49e810311f
Sep 18 19:04:03.298700 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.298675    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"]
Sep 18 19:04:03.971498 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.971456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562" event={"ID":"0c9d9d11-f274-47e3-acbc-ac6022077d14","Type":"ContainerStarted","Data":"e955e2c92db84b254954a3a92803f79301a71058320a6fc7b4c189a658647480"}
Sep 18 19:04:03.971992 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.971568    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562" event={"ID":"0c9d9d11-f274-47e3-acbc-ac6022077d14","Type":"ContainerStarted","Data":"d2b1d5599fe8bede337a67a086889f75b02eade816c99dec4d07a1e33482f0c1"}
Sep 18 19:04:03.971992 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.971601    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:03.973172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.973138    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg" event={"ID":"6a916c35-a3ac-4104-b559-5d3bb2ff160a","Type":"ContainerStarted","Data":"979fab9a7e9e40c02cc23e752e364db8f3d3ea7b2b1476ae15da4953efe9dfe6"}
Sep 18 19:04:03.973289 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.973194    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:03.973289 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.973219    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg" event={"ID":"6a916c35-a3ac-4104-b559-5d3bb2ff160a","Type":"ContainerStarted","Data":"56e4fde5944b41aeed25e010fc97d2319d03be5375d8e9cd969ecb49e810311f"}
Sep 18 19:04:03.990381 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:03.990323    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562" podStartSLOduration=1.990303876 podStartE2EDuration="1.990303876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:03.989472273 +0000 UTC m=+1173.427732476" watchObservedRunningTime="2026-09-18 19:04:03.990303876 +0000 UTC m=+1173.428564081"
Sep 18 19:04:04.012297 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:04.012243    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg" podStartSLOduration=2.01222591 podStartE2EDuration="2.01222591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:04:04.009658318 +0000 UTC m=+1173.447918522" watchObservedRunningTime="2026-09-18 19:04:04.01222591 +0000 UTC m=+1173.450486111"
Sep 18 19:04:11.445466 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.445429    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"]
Sep 18 19:04:11.449349 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.449327    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:11.452581 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.452561    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:04:11.452780 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.452767    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:04:11.458983 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.458961    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"]
Sep 18 19:04:11.585175 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.585132    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8b7343-4ea2-4eff-8692-9fdaab589a14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c\" (UID: \"5a8b7343-4ea2-4eff-8692-9fdaab589a14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:11.686004 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.685965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8b7343-4ea2-4eff-8692-9fdaab589a14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c\" (UID: \"5a8b7343-4ea2-4eff-8692-9fdaab589a14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:11.686661 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.686637    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8b7343-4ea2-4eff-8692-9fdaab589a14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c\" (UID: \"5a8b7343-4ea2-4eff-8692-9fdaab589a14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:11.763632 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:11.763592    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:12.039721 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:12.039695    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"]
Sep 18 19:04:12.041003 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:04:12.040980    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8b7343_4ea2_4eff_8692_9fdaab589a14.slice/crio-6294e1b0e9e22b585c85efa58da3ceba2591b80e578a02b070f156c18bb4d8de WatchSource:0}: Error finding container 6294e1b0e9e22b585c85efa58da3ceba2591b80e578a02b070f156c18bb4d8de: Status 404 returned error can't find the container with id 6294e1b0e9e22b585c85efa58da3ceba2591b80e578a02b070f156c18bb4d8de
Sep 18 19:04:13.037595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:13.037490    2568 generic.go:356] "Generic (PLEG): container finished" podID="5a8b7343-4ea2-4eff-8692-9fdaab589a14" containerID="4716f5092f65bba6329c25e79ca61844114ab7e3285461bae07e17befea9d49c" exitCode=0
Sep 18 19:04:13.037940 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:13.037588    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c" event={"ID":"5a8b7343-4ea2-4eff-8692-9fdaab589a14","Type":"ContainerDied","Data":"4716f5092f65bba6329c25e79ca61844114ab7e3285461bae07e17befea9d49c"}
Sep 18 19:04:13.037940 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:13.037638    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c" event={"ID":"5a8b7343-4ea2-4eff-8692-9fdaab589a14","Type":"ContainerStarted","Data":"6294e1b0e9e22b585c85efa58da3ceba2591b80e578a02b070f156c18bb4d8de"}
Sep 18 19:04:14.177733 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:14.177711    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:14.317943 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:14.317847    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8b7343-4ea2-4eff-8692-9fdaab589a14-maas-api-service-ca\") pod \"5a8b7343-4ea2-4eff-8692-9fdaab589a14\" (UID: \"5a8b7343-4ea2-4eff-8692-9fdaab589a14\") "
Sep 18 19:04:14.318263 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:14.318235    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a8b7343-4ea2-4eff-8692-9fdaab589a14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a8b7343-4ea2-4eff-8692-9fdaab589a14" (UID: "5a8b7343-4ea2-4eff-8692-9fdaab589a14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:14.419173 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:14.419144    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a8b7343-4ea2-4eff-8692-9fdaab589a14-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:14.982353 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:14.982324    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:14.982560 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:14.982437    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:15.054564 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:15.054510    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"
Sep 18 19:04:15.057005 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:15.056978    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c" event={"ID":"5a8b7343-4ea2-4eff-8692-9fdaab589a14","Type":"ContainerDied","Data":"6294e1b0e9e22b585c85efa58da3ceba2591b80e578a02b070f156c18bb4d8de"}
Sep 18 19:04:15.057165 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:15.057020    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6294e1b0e9e22b585c85efa58da3ceba2591b80e578a02b070f156c18bb4d8de"
Sep 18 19:04:27.896355 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:27.896313    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"]
Sep 18 19:04:27.896889 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:27.896599    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4" podUID="0f2a858c-912e-4bae-bc25-54636e0782df" containerName="payload-processing" containerID="cri-o://68b7e7033f5e08b76ad293b46e53091fdc39a61e13160758e83bd079983fa7c0" gracePeriod=30
Sep 18 19:04:27.907158 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:27.905355    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"]
Sep 18 19:04:27.907158 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:27.905625    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775" podUID="6755df83-18eb-452c-9339-ba32115574c5" containerName="payload-pre-processing" containerID="cri-o://5db9d04f1789fc8be98c265a42bb75bfb656648b57a48f578aa55e96c80899e8" gracePeriod=30
Sep 18 19:04:28.140185 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.140020    2568 generic.go:356] "Generic (PLEG): container finished" podID="6755df83-18eb-452c-9339-ba32115574c5" containerID="5db9d04f1789fc8be98c265a42bb75bfb656648b57a48f578aa55e96c80899e8" exitCode=0
Sep 18 19:04:28.140185 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.140112    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775" event={"ID":"6755df83-18eb-452c-9339-ba32115574c5","Type":"ContainerDied","Data":"5db9d04f1789fc8be98c265a42bb75bfb656648b57a48f578aa55e96c80899e8"}
Sep 18 19:04:28.142000 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.141981    2568 generic.go:356] "Generic (PLEG): container finished" podID="0f2a858c-912e-4bae-bc25-54636e0782df" containerID="68b7e7033f5e08b76ad293b46e53091fdc39a61e13160758e83bd079983fa7c0" exitCode=0
Sep 18 19:04:28.142260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.142139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4" event={"ID":"0f2a858c-912e-4bae-bc25-54636e0782df","Type":"ContainerDied","Data":"68b7e7033f5e08b76ad293b46e53091fdc39a61e13160758e83bd079983fa7c0"}
Sep 18 19:04:28.213566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.213540    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:04:28.253030 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.252997    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:04:28.276378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.276344    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"]
Sep 18 19:04:28.276598 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.276575    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2a858c-912e-4bae-bc25-54636e0782df-plugins-config-volume\") pod \"0f2a858c-912e-4bae-bc25-54636e0782df\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") "
Sep 18 19:04:28.276695 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.276678    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8srhl\" (UniqueName: \"kubernetes.io/projected/0f2a858c-912e-4bae-bc25-54636e0782df-kube-api-access-8srhl\") pod \"0f2a858c-912e-4bae-bc25-54636e0782df\" (UID: \"0f2a858c-912e-4bae-bc25-54636e0782df\") "
Sep 18 19:04:28.277043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.277008    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f2a858c-912e-4bae-bc25-54636e0782df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f2a858c-912e-4bae-bc25-54636e0782df" (UID: "0f2a858c-912e-4bae-bc25-54636e0782df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:28.280503 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.280471    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f2a858c-912e-4bae-bc25-54636e0782df-kube-api-access-8srhl" (OuterVolumeSpecName: "kube-api-access-8srhl") pod "0f2a858c-912e-4bae-bc25-54636e0782df" (UID: "0f2a858c-912e-4bae-bc25-54636e0782df"). InnerVolumeSpecName "kube-api-access-8srhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:28.286763 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.286627    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-cc7def-9zc5c"]
Sep 18 19:04:28.377973 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.377937    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbp46\" (UniqueName: \"kubernetes.io/projected/6755df83-18eb-452c-9339-ba32115574c5-kube-api-access-qbp46\") pod \"6755df83-18eb-452c-9339-ba32115574c5\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") "
Sep 18 19:04:28.378171 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.378121    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6755df83-18eb-452c-9339-ba32115574c5-plugins-config-volume\") pod \"6755df83-18eb-452c-9339-ba32115574c5\" (UID: \"6755df83-18eb-452c-9339-ba32115574c5\") "
Sep 18 19:04:28.378477 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.378452    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8srhl\" (UniqueName: \"kubernetes.io/projected/0f2a858c-912e-4bae-bc25-54636e0782df-kube-api-access-8srhl\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.378477 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.378473    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f2a858c-912e-4bae-bc25-54636e0782df-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.378680 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.378530    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6755df83-18eb-452c-9339-ba32115574c5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6755df83-18eb-452c-9339-ba32115574c5" (UID: "6755df83-18eb-452c-9339-ba32115574c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:28.380495 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.380467    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6755df83-18eb-452c-9339-ba32115574c5-kube-api-access-qbp46" (OuterVolumeSpecName: "kube-api-access-qbp46") pod "6755df83-18eb-452c-9339-ba32115574c5" (UID: "6755df83-18eb-452c-9339-ba32115574c5"). InnerVolumeSpecName "kube-api-access-qbp46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:28.480258 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.480215    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6755df83-18eb-452c-9339-ba32115574c5-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:28.480258 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:28.480253    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbp46\" (UniqueName: \"kubernetes.io/projected/6755df83-18eb-452c-9339-ba32115574c5-kube-api-access-qbp46\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:29.056328 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.056280    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8b7343-4ea2-4eff-8692-9fdaab589a14" path="/var/lib/kubelet/pods/5a8b7343-4ea2-4eff-8692-9fdaab589a14/volumes"
Sep 18 19:04:29.150379 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.150346    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"
Sep 18 19:04:29.150577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.150349    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775" event={"ID":"6755df83-18eb-452c-9339-ba32115574c5","Type":"ContainerDied","Data":"e498ccbfa636e66bb0c659b1c700e8def92fae7e073e463ef2096980ccc7eb82"}
Sep 18 19:04:29.150577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.150560    2568 scope.go:117] "RemoveContainer" containerID="5db9d04f1789fc8be98c265a42bb75bfb656648b57a48f578aa55e96c80899e8"
Sep 18 19:04:29.152364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.152297    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"
Sep 18 19:04:29.152493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.152300    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4" event={"ID":"0f2a858c-912e-4bae-bc25-54636e0782df","Type":"ContainerDied","Data":"33e1cdadbede8ecb72d35ffbbc7b0899b33c42891d986f5edf0cda6ad2f9c741"}
Sep 18 19:04:29.163176 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.163159    2568 scope.go:117] "RemoveContainer" containerID="68b7e7033f5e08b76ad293b46e53091fdc39a61e13160758e83bd079983fa7c0"
Sep 18 19:04:29.178932 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.178902    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"]
Sep 18 19:04:29.190158 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.190128    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-cc7def-cc46d9b98-p7775"]
Sep 18 19:04:29.227992 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.227959    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"]
Sep 18 19:04:29.233201 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:29.233177    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-cc7def-6bf559969c-tq6h4"]
Sep 18 19:04:30.536906 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.536833    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"]
Sep 18 19:04:30.537320 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537307    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6755df83-18eb-452c-9339-ba32115574c5" containerName="payload-pre-processing"
Sep 18 19:04:30.537364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537322    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6755df83-18eb-452c-9339-ba32115574c5" containerName="payload-pre-processing"
Sep 18 19:04:30.537364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537335    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f2a858c-912e-4bae-bc25-54636e0782df" containerName="payload-processing"
Sep 18 19:04:30.537364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537341    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f2a858c-912e-4bae-bc25-54636e0782df" containerName="payload-processing"
Sep 18 19:04:30.537364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537354    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8b7343-4ea2-4eff-8692-9fdaab589a14" containerName="revoke-keys"
Sep 18 19:04:30.537364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537359    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8b7343-4ea2-4eff-8692-9fdaab589a14" containerName="revoke-keys"
Sep 18 19:04:30.537547 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537443    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6755df83-18eb-452c-9339-ba32115574c5" containerName="payload-pre-processing"
Sep 18 19:04:30.537547 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537453    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f2a858c-912e-4bae-bc25-54636e0782df" containerName="payload-processing"
Sep 18 19:04:30.537547 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.537459    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8b7343-4ea2-4eff-8692-9fdaab589a14" containerName="revoke-keys"
Sep 18 19:04:30.539696 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.539681    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:30.542824 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.542805    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:04:30.546135 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.546108    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:04:30.562115 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.562089    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"]
Sep 18 19:04:30.602029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.601997    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm\" (UID: \"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:30.703540 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.703417    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm\" (UID: \"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:30.704031 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.704012    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm\" (UID: \"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:30.850444 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:30.850409    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:31.044500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.044469    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:04:31.048198 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.048173    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:04:31.057278 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.057248    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f2a858c-912e-4bae-bc25-54636e0782df" path="/var/lib/kubelet/pods/0f2a858c-912e-4bae-bc25-54636e0782df/volumes"
Sep 18 19:04:31.057760 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.057741    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6755df83-18eb-452c-9339-ba32115574c5" path="/var/lib/kubelet/pods/6755df83-18eb-452c-9339-ba32115574c5/volumes"
Sep 18 19:04:31.155849 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:04:31.155818    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f87763f_f4ca_44e6_ab6b_e23a7c0c5541.slice/crio-7ccf23a93cfe5ca55d154bbfc7239291efcf3e10cb51db72df25fafeb4e6d5ee WatchSource:0}: Error finding container 7ccf23a93cfe5ca55d154bbfc7239291efcf3e10cb51db72df25fafeb4e6d5ee: Status 404 returned error can't find the container with id 7ccf23a93cfe5ca55d154bbfc7239291efcf3e10cb51db72df25fafeb4e6d5ee
Sep 18 19:04:31.156382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.156347    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"]
Sep 18 19:04:31.168122 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.168094    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm" event={"ID":"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541","Type":"ContainerStarted","Data":"7ccf23a93cfe5ca55d154bbfc7239291efcf3e10cb51db72df25fafeb4e6d5ee"}
Sep 18 19:04:31.169770 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.169750    2568 generic.go:356] "Generic (PLEG): container finished" podID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerID="7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e" exitCode=0
Sep 18 19:04:31.169879 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.169825    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" event={"ID":"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7","Type":"ContainerDied","Data":"7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"}
Sep 18 19:04:31.170275 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.170255    2568 scope.go:117] "RemoveContainer" containerID="7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"
Sep 18 19:04:31.171794 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.171763    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:04:31.171878 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.171830    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:04:31.171878 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:31.171868    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:04:32.185435 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:32.185398    2568 generic.go:356] "Generic (PLEG): container finished" podID="7f87763f-f4ca-44e6-ab6b-e23a7c0c5541" containerID="080840821ec9404de7d61bc6652cf6be3c8b77dea14fda7ec18703774571aef8" exitCode=0
Sep 18 19:04:32.185917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:32.185488    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm" event={"ID":"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541","Type":"ContainerDied","Data":"080840821ec9404de7d61bc6652cf6be3c8b77dea14fda7ec18703774571aef8"}
Sep 18 19:04:32.187638 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:32.187610    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" event={"ID":"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7","Type":"ContainerStarted","Data":"7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb"}
Sep 18 19:04:32.639635 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:32.639592    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:32.640494 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:32.640460    2568 patch_prober.go:28] interesting pod/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:04:32.640626 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:32.640560    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 18 19:04:33.380712 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:33.380686    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:33.547002 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:33.546947    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541-maas-api-service-ca\") pod \"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541\" (UID: \"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541\") "
Sep 18 19:04:33.547806 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:33.547770    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f87763f-f4ca-44e6-ab6b-e23a7c0c5541" (UID: "7f87763f-f4ca-44e6-ab6b-e23a7c0c5541"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:33.640540 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:33.640204    2568 patch_prober.go:28] interesting pod/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:04:33.640540 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:33.640285    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 18 19:04:33.649012 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:33.648969    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:34.200604 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:34.200573    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"
Sep 18 19:04:34.200805 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:34.200608    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm" event={"ID":"7f87763f-f4ca-44e6-ab6b-e23a7c0c5541","Type":"ContainerDied","Data":"7ccf23a93cfe5ca55d154bbfc7239291efcf3e10cb51db72df25fafeb4e6d5ee"}
Sep 18 19:04:34.200805 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:34.200653    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ccf23a93cfe5ca55d154bbfc7239291efcf3e10cb51db72df25fafeb4e6d5ee"
Sep 18 19:04:34.640447 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:34.639970    2568 patch_prober.go:28] interesting pod/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:04:34.640447 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:34.640041    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 18 19:04:34.888240 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:34.888209    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:35.640306 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:35.640266    2568 patch_prober.go:28] interesting pod/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:04:35.640636 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:35.640606    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 18 19:04:36.639569 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:36.639498    2568 patch_prober.go:28] interesting pod/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 18 19:04:36.639885 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:36.639854    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 18 19:04:37.643590 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:37.643555    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:37.644741 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:37.644719    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:41.914163 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:41.914128    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"]
Sep 18 19:04:41.914645 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:41.914407    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg" podUID="6a916c35-a3ac-4104-b559-5d3bb2ff160a" containerName="payload-processing" containerID="cri-o://979fab9a7e9e40c02cc23e752e364db8f3d3ea7b2b1476ae15da4953efe9dfe6" gracePeriod=30
Sep 18 19:04:41.938792 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:41.938760    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"]
Sep 18 19:04:41.939067 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:41.939006    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562" podUID="0c9d9d11-f274-47e3-acbc-ac6022077d14" containerName="payload-pre-processing" containerID="cri-o://e955e2c92db84b254954a3a92803f79301a71058320a6fc7b4c189a658647480" gracePeriod=30
Sep 18 19:04:42.181315 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.180991    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"]
Sep 18 19:04:42.185179 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.185126    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fa6711c6-2jvtm"]
Sep 18 19:04:42.252141 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.252108    2568 generic.go:356] "Generic (PLEG): container finished" podID="0c9d9d11-f274-47e3-acbc-ac6022077d14" containerID="e955e2c92db84b254954a3a92803f79301a71058320a6fc7b4c189a658647480" exitCode=0
Sep 18 19:04:42.252281 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.252183    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562" event={"ID":"0c9d9d11-f274-47e3-acbc-ac6022077d14","Type":"ContainerDied","Data":"e955e2c92db84b254954a3a92803f79301a71058320a6fc7b4c189a658647480"}
Sep 18 19:04:42.254040 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.254015    2568 generic.go:356] "Generic (PLEG): container finished" podID="6a916c35-a3ac-4104-b559-5d3bb2ff160a" containerID="979fab9a7e9e40c02cc23e752e364db8f3d3ea7b2b1476ae15da4953efe9dfe6" exitCode=0
Sep 18 19:04:42.254169 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.254074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg" event={"ID":"6a916c35-a3ac-4104-b559-5d3bb2ff160a","Type":"ContainerDied","Data":"979fab9a7e9e40c02cc23e752e364db8f3d3ea7b2b1476ae15da4953efe9dfe6"}
Sep 18 19:04:42.254169 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.254119    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg" event={"ID":"6a916c35-a3ac-4104-b559-5d3bb2ff160a","Type":"ContainerDied","Data":"56e4fde5944b41aeed25e010fc97d2319d03be5375d8e9cd969ecb49e810311f"}
Sep 18 19:04:42.254169 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.254142    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56e4fde5944b41aeed25e010fc97d2319d03be5375d8e9cd969ecb49e810311f"
Sep 18 19:04:42.261718 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.261701    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:42.264845 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.264827    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:42.342776 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.342739    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b64lb\" (UniqueName: \"kubernetes.io/projected/6a916c35-a3ac-4104-b559-5d3bb2ff160a-kube-api-access-b64lb\") pod \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") "
Sep 18 19:04:42.342952 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.342809    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d9d11-f274-47e3-acbc-ac6022077d14-plugins-config-volume\") pod \"0c9d9d11-f274-47e3-acbc-ac6022077d14\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") "
Sep 18 19:04:42.342952 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.342864    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47rrx\" (UniqueName: \"kubernetes.io/projected/0c9d9d11-f274-47e3-acbc-ac6022077d14-kube-api-access-47rrx\") pod \"0c9d9d11-f274-47e3-acbc-ac6022077d14\" (UID: \"0c9d9d11-f274-47e3-acbc-ac6022077d14\") "
Sep 18 19:04:42.342952 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.342933    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a916c35-a3ac-4104-b559-5d3bb2ff160a-plugins-config-volume\") pod \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\" (UID: \"6a916c35-a3ac-4104-b559-5d3bb2ff160a\") "
Sep 18 19:04:42.343229 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.343192    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9d9d11-f274-47e3-acbc-ac6022077d14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c9d9d11-f274-47e3-acbc-ac6022077d14" (UID: "0c9d9d11-f274-47e3-acbc-ac6022077d14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:42.343335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.343257    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a916c35-a3ac-4104-b559-5d3bb2ff160a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a916c35-a3ac-4104-b559-5d3bb2ff160a" (UID: "6a916c35-a3ac-4104-b559-5d3bb2ff160a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:42.343402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.343383    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9d9d11-f274-47e3-acbc-ac6022077d14-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.343402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.343400    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a916c35-a3ac-4104-b559-5d3bb2ff160a-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.344867 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.344851    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a916c35-a3ac-4104-b559-5d3bb2ff160a-kube-api-access-b64lb" (OuterVolumeSpecName: "kube-api-access-b64lb") pod "6a916c35-a3ac-4104-b559-5d3bb2ff160a" (UID: "6a916c35-a3ac-4104-b559-5d3bb2ff160a"). InnerVolumeSpecName "kube-api-access-b64lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:42.345451 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.345431    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9d9d11-f274-47e3-acbc-ac6022077d14-kube-api-access-47rrx" (OuterVolumeSpecName: "kube-api-access-47rrx") pod "0c9d9d11-f274-47e3-acbc-ac6022077d14" (UID: "0c9d9d11-f274-47e3-acbc-ac6022077d14"). InnerVolumeSpecName "kube-api-access-47rrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:42.444418 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.444332    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b64lb\" (UniqueName: \"kubernetes.io/projected/6a916c35-a3ac-4104-b559-5d3bb2ff160a-kube-api-access-b64lb\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:42.444418 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:42.444360    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47rrx\" (UniqueName: \"kubernetes.io/projected/0c9d9d11-f274-47e3-acbc-ac6022077d14-kube-api-access-47rrx\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:43.054954 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.054918    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f87763f-f4ca-44e6-ab6b-e23a7c0c5541" path="/var/lib/kubelet/pods/7f87763f-f4ca-44e6-ab6b-e23a7c0c5541/volumes"
Sep 18 19:04:43.261675 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.261643    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"
Sep 18 19:04:43.261849 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.261676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562" event={"ID":"0c9d9d11-f274-47e3-acbc-ac6022077d14","Type":"ContainerDied","Data":"d2b1d5599fe8bede337a67a086889f75b02eade816c99dec4d07a1e33482f0c1"}
Sep 18 19:04:43.261849 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.261741    2568 scope.go:117] "RemoveContainer" containerID="e955e2c92db84b254954a3a92803f79301a71058320a6fc7b4c189a658647480"
Sep 18 19:04:43.261849 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.261782    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"
Sep 18 19:04:43.321707 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.321674    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"]
Sep 18 19:04:43.323798 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.323777    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fa6711c6-6ff6b757cc-4qjkg"]
Sep 18 19:04:43.354261 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.354224    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"]
Sep 18 19:04:43.359968 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:43.359938    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fa6711c6-7587c5d7fd-7v562"]
Sep 18 19:04:45.055430 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:45.055398    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9d9d11-f274-47e3-acbc-ac6022077d14" path="/var/lib/kubelet/pods/0c9d9d11-f274-47e3-acbc-ac6022077d14/volumes"
Sep 18 19:04:45.055841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:45.055800    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a916c35-a3ac-4104-b559-5d3bb2ff160a" path="/var/lib/kubelet/pods/6a916c35-a3ac-4104-b559-5d3bb2ff160a/volumes"
Sep 18 19:04:50.643133 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:50.643098    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"]
Sep 18 19:04:50.643576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:50.643340    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" containerID="cri-o://7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb" gracePeriod=30
Sep 18 19:04:52.646290 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:52.646255    2568 patch_prober.go:28] interesting pod/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:04:52.646679 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:52.646313    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:04:55.908877 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.908819    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:55.979670 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979642    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-socket\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.979848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979740    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-certs\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.979848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979778    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istiod-ca-cert\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.979848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979794    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-data\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.979848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979817    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-credential-socket\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.979848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979840    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-podinfo\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.980113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979905    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-token\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.980113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979955    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54vrt\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-kube-api-access-54vrt\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.980113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979979    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:55.980113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.979992    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-envoy\") pod \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\" (UID: \"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7\") "
Sep 18 19:04:55.980113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.980092    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:55.980714 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.980420    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:55.980714 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.980475    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:55.980714 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.980483    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:55.980714 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.980502    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:55.980714 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.980546    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:04:55.982792 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.982763    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:04:55.982903 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.982791    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-kube-api-access-54vrt" (OuterVolumeSpecName: "kube-api-access-54vrt") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "kube-api-access-54vrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:55.982903 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.982803    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:04:55.982903 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:55.982808    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" (UID: "63ef8e4c-71d2-4f14-83c9-b7655a94b2a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:04:56.081256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081224    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.081256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081251    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.081256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081260    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.081476 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081268    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.081476 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081277    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.081476 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081285    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54vrt\" (UniqueName: \"kubernetes.io/projected/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-kube-api-access-54vrt\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.081476 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.081293    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:04:56.346562 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.346506    2568 generic.go:356] "Generic (PLEG): container finished" podID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerID="7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb" exitCode=0
Sep 18 19:04:56.346773 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.346568    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" event={"ID":"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7","Type":"ContainerDied","Data":"7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb"}
Sep 18 19:04:56.346773 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.346582    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"
Sep 18 19:04:56.346773 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.346620    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5" event={"ID":"63ef8e4c-71d2-4f14-83c9-b7655a94b2a7","Type":"ContainerDied","Data":"beca53648cba4176cc12b09ae299740d9a679b30f3ef7a7bd39f9fdd88450ee1"}
Sep 18 19:04:56.346773 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.346650    2568 scope.go:117] "RemoveContainer" containerID="7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb"
Sep 18 19:04:56.356802 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.356786    2568 scope.go:117] "RemoveContainer" containerID="7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"
Sep 18 19:04:56.365475 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.365457    2568 scope.go:117] "RemoveContainer" containerID="7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb"
Sep 18 19:04:56.365741 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:04:56.365724    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb\": container with ID starting with 7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb not found: ID does not exist" containerID="7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb"
Sep 18 19:04:56.365796 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.365749    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb"} err="failed to get container status \"7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb\": rpc error: code = NotFound desc = could not find container \"7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb\": container with ID starting with 7744c5eb341c76743c14c9f1c1ed75cd219cbe277758e1e65f07eb2fb9b537cb not found: ID does not exist"
Sep 18 19:04:56.365796 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.365767    2568 scope.go:117] "RemoveContainer" containerID="7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"
Sep 18 19:04:56.366039 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:04:56.366021    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e\": container with ID starting with 7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e not found: ID does not exist" containerID="7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"
Sep 18 19:04:56.366084 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.366046    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e"} err="failed to get container status \"7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e\": rpc error: code = NotFound desc = could not find container \"7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e\": container with ID starting with 7b91ae6109ae0c2e04a9317eef2eeb186daabb1b1e91ea56d5c746992064f36e not found: ID does not exist"
Sep 18 19:04:56.387832 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.387807    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"]
Sep 18 19:04:56.397839 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:56.397815    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fa6711c6-openshift-default-748fb7d48-9zqf5"]
Sep 18 19:04:57.055363 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:04:57.055328    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" path="/var/lib/kubelet/pods/63ef8e4c-71d2-4f14-83c9-b7655a94b2a7/volumes"
Sep 18 19:05:07.938564 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.938497    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"]
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.938956    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a916c35-a3ac-4104-b559-5d3bb2ff160a" containerName="payload-processing"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.938970    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a916c35-a3ac-4104-b559-5d3bb2ff160a" containerName="payload-processing"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.938992    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9d9d11-f274-47e3-acbc-ac6022077d14" containerName="payload-pre-processing"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939000    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9d9d11-f274-47e3-acbc-ac6022077d14" containerName="payload-pre-processing"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939017    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939023    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939038    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939046    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939056    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f87763f-f4ca-44e6-ab6b-e23a7c0c5541" containerName="revoke-keys"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939067    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f87763f-f4ca-44e6-ab6b-e23a7c0c5541" containerName="revoke-keys"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939158    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9d9d11-f274-47e3-acbc-ac6022077d14" containerName="payload-pre-processing"
Sep 18 19:05:07.939164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939169    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy"
Sep 18 19:05:07.939674 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939177    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="63ef8e4c-71d2-4f14-83c9-b7655a94b2a7" containerName="istio-proxy"
Sep 18 19:05:07.939674 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939185    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f87763f-f4ca-44e6-ab6b-e23a7c0c5541" containerName="revoke-keys"
Sep 18 19:05:07.939674 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.939216    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a916c35-a3ac-4104-b559-5d3bb2ff160a" containerName="payload-processing"
Sep 18 19:05:07.942599 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.942581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:07.949789 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:07.949771    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-3b5822-openshift-default-dockercfg-q67zh\""
Sep 18 19:05:08.090329 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.090296    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"]
Sep 18 19:05:08.103651 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103615    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istiod-ca-cert\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.103651 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-credential-socket\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.103902 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103678    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-podinfo\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.103902 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103694    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-socket\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.103902 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103787    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-data\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.103902 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103894    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-certs\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.104129 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103922    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-token\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.104129 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.103943    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-envoy\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.104129 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.104008    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hgp2\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-kube-api-access-7hgp2\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.204926 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.204888    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-certs\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.204926 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.204928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-token\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.204946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-envoy\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.204976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hgp2\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-kube-api-access-7hgp2\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istiod-ca-cert\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205085    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-credential-socket\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205147 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205133    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-podinfo\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205445 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205174    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-socket\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205445 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205248    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-data\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-data\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205803 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205641    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-credential-socket\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205803 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-certs\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.205803 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istiod-ca-cert\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.206009 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.205770    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-socket\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.207267 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.207246    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-envoy\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.207806 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.207789    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-podinfo\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.217743 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.217725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hgp2\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-kube-api-access-7hgp2\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.226369 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.226350    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-token\") pod \"e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") " pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.252554 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.252504    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:08.549946 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.549918    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"]
Sep 18 19:05:08.551510 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:05:08.551479    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6fe400_5f1d_4c17_896d_e1332effb9c4.slice/crio-f5f58ac951ca0943a93d6127957e50913743043d10574d9d79e3f87d463801af WatchSource:0}: Error finding container f5f58ac951ca0943a93d6127957e50913743043d10574d9d79e3f87d463801af: Status 404 returned error can't find the container with id f5f58ac951ca0943a93d6127957e50913743043d10574d9d79e3f87d463801af
Sep 18 19:05:08.553742 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.553708    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:08.554098 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.553772    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:08.554098 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:08.553801    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:09.432448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:09.432417    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" event={"ID":"7f6fe400-5f1d-4c17-896d-e1332effb9c4","Type":"ContainerStarted","Data":"5d82bb65ed240b6817a4ba2d47e46de4e98627c40f00f43d35b8d347d4f207de"}
Sep 18 19:05:09.432846 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:09.432465    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" event={"ID":"7f6fe400-5f1d-4c17-896d-e1332effb9c4","Type":"ContainerStarted","Data":"f5f58ac951ca0943a93d6127957e50913743043d10574d9d79e3f87d463801af"}
Sep 18 19:05:09.479037 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:09.478975    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podStartSLOduration=2.478955698 podStartE2EDuration="2.478955698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:09.468539767 +0000 UTC m=+1238.906799971" watchObservedRunningTime="2026-09-18 19:05:09.478955698 +0000 UTC m=+1238.917215902"
Sep 18 19:05:10.253086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:10.253041    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:10.258700 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:10.258673    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:10.439125 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:10.439095    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:10.439985 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:10.439967    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:14.532412 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.532377    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"]
Sep 18 19:05:14.536384 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.536355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.540589 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.540568    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-9674d9-openshift-default-dockercfg-c6cnm\""
Sep 18 19:05:14.554504 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.554473    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"]
Sep 18 19:05:14.677324 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677284    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-certs\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677324 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677329    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-envoy\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677393    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-podinfo\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677579    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-credential-socket\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677631    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-token\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677857 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677667    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-socket\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677857 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677767    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istiod-ca-cert\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677857 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-data\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.677857 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.677839    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58cmd\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-kube-api-access-58cmd\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.778852 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.778796    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58cmd\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-kube-api-access-58cmd\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.778886    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-certs\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.778923    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-envoy\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.778951    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-podinfo\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-credential-socket\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779119    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-token\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779149    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-socket\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istiod-ca-cert\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779243    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-data\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779742 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779715    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-data\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.779916 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.779888    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-certs\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.780862 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.780147    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-credential-socket\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.780862 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.780774    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istiod-ca-cert\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.781164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.781133    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-socket\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.783491 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.783430    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-envoy\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.783726 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.783706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-podinfo\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.802411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.799428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58cmd\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-kube-api-access-58cmd\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.802411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.802277    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-token\") pod \"e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") " pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:14.847950 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:14.847914    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:15.169912 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.169833    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"]
Sep 18 19:05:15.174390 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:05:15.174362    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc69e4b0_57df_4ecb_a224_daddfbe42efd.slice/crio-4d7100823ad1dd799f126cde8edbfc5040029e13123ff81e31db4808e6d8b413 WatchSource:0}: Error finding container 4d7100823ad1dd799f126cde8edbfc5040029e13123ff81e31db4808e6d8b413: Status 404 returned error can't find the container with id 4d7100823ad1dd799f126cde8edbfc5040029e13123ff81e31db4808e6d8b413
Sep 18 19:05:15.176790 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.176754    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:15.176869 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.176820    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:15.176869 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.176849    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:15.468560 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.468502    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49" event={"ID":"dc69e4b0-57df-4ecb-a224-daddfbe42efd","Type":"ContainerStarted","Data":"30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d"}
Sep 18 19:05:15.468724 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.468589    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49" event={"ID":"dc69e4b0-57df-4ecb-a224-daddfbe42efd","Type":"ContainerStarted","Data":"4d7100823ad1dd799f126cde8edbfc5040029e13123ff81e31db4808e6d8b413"}
Sep 18 19:05:15.515953 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.515905    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49" podStartSLOduration=1.515889279 podStartE2EDuration="1.515889279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:15.508869493 +0000 UTC m=+1244.947129698" watchObservedRunningTime="2026-09-18 19:05:15.515889279 +0000 UTC m=+1244.954149474"
Sep 18 19:05:15.848246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:15.848169    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:16.853706 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:16.853673    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:17.480893 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:17.480862    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:17.481880 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:17.481860    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:05:18.819351 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.819303    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"]
Sep 18 19:05:18.824365 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.824338    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:18.828973 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.828948    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-3b5822\""
Sep 18 19:05:18.829172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.829113    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-3b5822-dockercfg-6fp4c\""
Sep 18 19:05:18.843279 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.843249    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"]
Sep 18 19:05:18.847633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.847605    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"]
Sep 18 19:05:18.851409 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.851387    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:18.870340 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.870316    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"]
Sep 18 19:05:18.926983 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.926946    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmzng\" (UniqueName: \"kubernetes.io/projected/5e2bae64-e619-40db-967a-0ac0fe1216a0-kube-api-access-mmzng\") pod \"payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:18.927633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.927256    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:18.927766 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.927677    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppg9z\" (UniqueName: \"kubernetes.io/projected/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-kube-api-access-ppg9z\") pod \"payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:18.927766 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:18.927754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2bae64-e619-40db-967a-0ac0fe1216a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:19.029327 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.029289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmzng\" (UniqueName: \"kubernetes.io/projected/5e2bae64-e619-40db-967a-0ac0fe1216a0-kube-api-access-mmzng\") pod \"payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:19.029327 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.029334    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:19.029567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.029366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppg9z\" (UniqueName: \"kubernetes.io/projected/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-kube-api-access-ppg9z\") pod \"payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:19.029567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.029414    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2bae64-e619-40db-967a-0ac0fe1216a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:19.030119 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.030098    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2bae64-e619-40db-967a-0ac0fe1216a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:19.030119 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.030110    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:19.040193 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.040164    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppg9z\" (UniqueName: \"kubernetes.io/projected/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-kube-api-access-ppg9z\") pod \"payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:19.040333 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.040307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmzng\" (UniqueName: \"kubernetes.io/projected/5e2bae64-e619-40db-967a-0ac0fe1216a0-kube-api-access-mmzng\") pod \"payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:19.145450 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.145347    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:19.161886 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.161859    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:19.433113 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.433036    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"]
Sep 18 19:05:19.436672 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:05:19.436641    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2bae64_e619_40db_967a_0ac0fe1216a0.slice/crio-2eea2614262e2535defeb7a42f147d5f7044e4117c6162ccca2cc6c138340cdf WatchSource:0}: Error finding container 2eea2614262e2535defeb7a42f147d5f7044e4117c6162ccca2cc6c138340cdf: Status 404 returned error can't find the container with id 2eea2614262e2535defeb7a42f147d5f7044e4117c6162ccca2cc6c138340cdf
Sep 18 19:05:19.454651 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.454626    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"]
Sep 18 19:05:19.456167 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:05:19.456144    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3522ed84_bc0d_4a8b_a5b6_665847aa6a4a.slice/crio-7d97b147f0f51c46cfef53832edbc5a4318ac6def0f2ba90b63636673b2612ba WatchSource:0}: Error finding container 7d97b147f0f51c46cfef53832edbc5a4318ac6def0f2ba90b63636673b2612ba: Status 404 returned error can't find the container with id 7d97b147f0f51c46cfef53832edbc5a4318ac6def0f2ba90b63636673b2612ba
Sep 18 19:05:19.493009 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.492972    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8" event={"ID":"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a","Type":"ContainerStarted","Data":"7d97b147f0f51c46cfef53832edbc5a4318ac6def0f2ba90b63636673b2612ba"}
Sep 18 19:05:19.494675 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:19.494642    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws" event={"ID":"5e2bae64-e619-40db-967a-0ac0fe1216a0","Type":"ContainerStarted","Data":"2eea2614262e2535defeb7a42f147d5f7044e4117c6162ccca2cc6c138340cdf"}
Sep 18 19:05:20.501931 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:20.501891    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8" event={"ID":"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a","Type":"ContainerStarted","Data":"4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b"}
Sep 18 19:05:20.502373 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:20.501997    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:20.503487 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:20.503463    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws" event={"ID":"5e2bae64-e619-40db-967a-0ac0fe1216a0","Type":"ContainerStarted","Data":"86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c"}
Sep 18 19:05:20.503648 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:20.503631    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:20.553031 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:20.552986    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8" podStartSLOduration=2.552971989 podStartE2EDuration="2.552971989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:20.550130903 +0000 UTC m=+1249.988391120" watchObservedRunningTime="2026-09-18 19:05:20.552971989 +0000 UTC m=+1249.991232193"
Sep 18 19:05:20.626217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:20.626158    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws" podStartSLOduration=2.626139178 podStartE2EDuration="2.626139178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:20.625020297 +0000 UTC m=+1250.063280501" watchObservedRunningTime="2026-09-18 19:05:20.626139178 +0000 UTC m=+1250.064399384"
Sep 18 19:05:31.511732 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:31.511701    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:05:31.512219 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:31.511780    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:05:40.713655 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:40.713582    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": read tcp 10.133.0.2:55712->10.133.0.63:15021: read: connection reset by peer" start-of-body=
Sep 18 19:05:40.714081 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:40.713666    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": read tcp 10.133.0.2:55712->10.133.0.63:15021: read: connection reset by peer"
Sep 18 19:05:41.648663 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:41.648621    2568 generic.go:356] "Generic (PLEG): container finished" podID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerID="5d82bb65ed240b6817a4ba2d47e46de4e98627c40f00f43d35b8d347d4f207de" exitCode=0
Sep 18 19:05:41.648663 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:41.648652    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" event={"ID":"7f6fe400-5f1d-4c17-896d-e1332effb9c4","Type":"ContainerDied","Data":"5d82bb65ed240b6817a4ba2d47e46de4e98627c40f00f43d35b8d347d4f207de"}
Sep 18 19:05:41.649190 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:41.649176    2568 scope.go:117] "RemoveContainer" containerID="5d82bb65ed240b6817a4ba2d47e46de4e98627c40f00f43d35b8d347d4f207de"
Sep 18 19:05:41.650425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:41.650394    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:41.650532 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:41.650463    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:41.650532 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:41.650496    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:05:42.656899 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:42.656867    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" event={"ID":"7f6fe400-5f1d-4c17-896d-e1332effb9c4","Type":"ContainerStarted","Data":"568f4e22fe22bc1d495712c75af74f52ae755d0acea149c831970dbf29fe4afd"}
Sep 18 19:05:43.253651 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:43.253614    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:43.253901 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:43.253883    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 18 19:05:43.253948 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:43.253925    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 18 19:05:44.253439 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:44.253401    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 18 19:05:44.253871 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:44.253464    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 18 19:05:45.255284 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:45.254810    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 18 19:05:45.255284 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:45.254875    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 18 19:05:46.253919 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:46.253884    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 18 19:05:46.254095 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:46.253938    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 18 19:05:47.253891 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:47.253851    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 18 19:05:47.254362 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:47.253924    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 18 19:05:48.253092 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:48.253053    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 18 19:05:48.253299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:48.253119    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 18 19:05:49.257285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:49.257254    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:49.257765 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:49.257600    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:49.258336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:49.258316    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:05:51.283915 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.283880    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"]
Sep 18 19:05:51.287480 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.287455    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.290051 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.290032    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-4805a7\""
Sep 18 19:05:51.290168 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.290075    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-4805a7-dockercfg-xr8zt\""
Sep 18 19:05:51.307375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.307349    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"]
Sep 18 19:05:51.389462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.389428    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9rkg\" (UniqueName: \"kubernetes.io/projected/099537b0-bc73-4c11-8905-6424c25e40b0-kube-api-access-d9rkg\") pod \"payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.389672 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.389506    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099537b0-bc73-4c11-8905-6424c25e40b0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.490891 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.490853    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9rkg\" (UniqueName: \"kubernetes.io/projected/099537b0-bc73-4c11-8905-6424c25e40b0-kube-api-access-d9rkg\") pod \"payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.491064 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.490941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099537b0-bc73-4c11-8905-6424c25e40b0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.491699 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.491680    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099537b0-bc73-4c11-8905-6424c25e40b0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.500328 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.500301    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9rkg\" (UniqueName: \"kubernetes.io/projected/099537b0-bc73-4c11-8905-6424c25e40b0-kube-api-access-d9rkg\") pod \"payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.597821 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.597722    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:05:51.892861 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:51.892829    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"]
Sep 18 19:05:51.894115 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:05:51.894088    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099537b0_bc73_4c11_8905_6424c25e40b0.slice/crio-f7f62740fd51e36e347c0411d781eb60e603d00386101973328fadc8481f5d1e WatchSource:0}: Error finding container f7f62740fd51e36e347c0411d781eb60e603d00386101973328fadc8481f5d1e: Status 404 returned error can't find the container with id f7f62740fd51e36e347c0411d781eb60e603d00386101973328fadc8481f5d1e
Sep 18 19:05:52.720658 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:52.720619    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5" event={"ID":"099537b0-bc73-4c11-8905-6424c25e40b0","Type":"ContainerStarted","Data":"70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c"}
Sep 18 19:05:52.721047 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:52.720684    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5" event={"ID":"099537b0-bc73-4c11-8905-6424c25e40b0","Type":"ContainerStarted","Data":"f7f62740fd51e36e347c0411d781eb60e603d00386101973328fadc8481f5d1e"}
Sep 18 19:05:52.721047 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:05:52.720727    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:06:03.729557 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:03.729488    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:06:03.750746 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:03.750691    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5" podStartSLOduration=12.750672646 podStartE2EDuration="12.750672646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:05:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:05:52.769805085 +0000 UTC m=+1282.208065290" watchObservedRunningTime="2026-09-18 19:06:03.750672646 +0000 UTC m=+1293.188932851"
Sep 18 19:06:09.130371 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.130332    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"]
Sep 18 19:06:09.133068 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.133052    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:09.137016 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.136988    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:06:09.137722 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.137703    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:06:09.158377 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.158334    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"]
Sep 18 19:06:09.288226 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.288188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd\" (UID: \"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:09.389111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.389026    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd\" (UID: \"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:09.389727 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.389705    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd\" (UID: \"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:09.444026 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.443994    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:09.777009 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.776974    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"]
Sep 18 19:06:09.784299 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:06:09.784270    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3e2803_05a3_4563_a6d5_ab8f411bf9ef.slice/crio-e941c3bb6b64f6ef9f09c70df64018ad4373a7af26ad2719ca9841ade268ca0c WatchSource:0}: Error finding container e941c3bb6b64f6ef9f09c70df64018ad4373a7af26ad2719ca9841ade268ca0c: Status 404 returned error can't find the container with id e941c3bb6b64f6ef9f09c70df64018ad4373a7af26ad2719ca9841ade268ca0c
Sep 18 19:06:09.831933 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:09.831900    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd" event={"ID":"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef","Type":"ContainerStarted","Data":"e941c3bb6b64f6ef9f09c70df64018ad4373a7af26ad2719ca9841ade268ca0c"}
Sep 18 19:06:10.839343 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:10.839242    2568 generic.go:356] "Generic (PLEG): container finished" podID="dc3e2803-05a3-4563-a6d5-ab8f411bf9ef" containerID="5a5c5a073812ee7d9cd144b9b9007b908d3eebbea45adbf3898820b9a24360e5" exitCode=0
Sep 18 19:06:10.839343 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:10.839328    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd" event={"ID":"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef","Type":"ContainerDied","Data":"5a5c5a073812ee7d9cd144b9b9007b908d3eebbea45adbf3898820b9a24360e5"}
Sep 18 19:06:11.992402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:11.992378    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:12.116551 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:12.116450    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef-maas-api-service-ca\") pod \"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef\" (UID: \"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef\") "
Sep 18 19:06:12.116825 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:12.116804    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc3e2803-05a3-4563-a6d5-ab8f411bf9ef" (UID: "dc3e2803-05a3-4563-a6d5-ab8f411bf9ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:12.218128 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:12.218091    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:12.852212 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:12.852182    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"
Sep 18 19:06:12.852398 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:12.852212    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd" event={"ID":"dc3e2803-05a3-4563-a6d5-ab8f411bf9ef","Type":"ContainerDied","Data":"e941c3bb6b64f6ef9f09c70df64018ad4373a7af26ad2719ca9841ade268ca0c"}
Sep 18 19:06:12.852398 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:12.852257    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e941c3bb6b64f6ef9f09c70df64018ad4373a7af26ad2719ca9841ade268ca0c"
Sep 18 19:06:27.102490 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.102453    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"]
Sep 18 19:06:27.110016 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.109990    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-15c3ed-29829300-fzv2h"]
Sep 18 19:06:27.160112 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.160081    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"]
Sep 18 19:06:27.160319 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.160278    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k" podUID="a060e899-ee44-4687-9d30-e28f0be7a279" containerName="payload-processing" containerID="cri-o://93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990" gracePeriod=30
Sep 18 19:06:27.185880 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.185849    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"]
Sep 18 19:06:27.186167 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.186101    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x" podUID="4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" containerName="payload-pre-processing" containerID="cri-o://253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd" gracePeriod=30
Sep 18 19:06:27.446082 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.446052    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"]
Sep 18 19:06:27.453190 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.452961    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-15c3ed-4fmzd"]
Sep 18 19:06:27.478090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.478066    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 19:06:27.481201 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.481180    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 19:06:27.585744 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.585707    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7znj\" (UniqueName: \"kubernetes.io/projected/a060e899-ee44-4687-9d30-e28f0be7a279-kube-api-access-p7znj\") pod \"a060e899-ee44-4687-9d30-e28f0be7a279\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") "
Sep 18 19:06:27.585931 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.585870    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a060e899-ee44-4687-9d30-e28f0be7a279-plugins-config-volume\") pod \"a060e899-ee44-4687-9d30-e28f0be7a279\" (UID: \"a060e899-ee44-4687-9d30-e28f0be7a279\") "
Sep 18 19:06:27.586010 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.585986    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-plugins-config-volume\") pod \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") "
Sep 18 19:06:27.586073 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.586038    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82ntr\" (UniqueName: \"kubernetes.io/projected/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-kube-api-access-82ntr\") pod \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\" (UID: \"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23\") "
Sep 18 19:06:27.586543 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.586355    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a060e899-ee44-4687-9d30-e28f0be7a279-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a060e899-ee44-4687-9d30-e28f0be7a279" (UID: "a060e899-ee44-4687-9d30-e28f0be7a279"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:27.586671 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.586429    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" (UID: "4850a56b-3199-4b8a-af1d-1c1e4e3d5b23"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:27.586837 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.586818    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a060e899-ee44-4687-9d30-e28f0be7a279-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.586904 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.586839    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.588014 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.587988    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a060e899-ee44-4687-9d30-e28f0be7a279-kube-api-access-p7znj" (OuterVolumeSpecName: "kube-api-access-p7znj") pod "a060e899-ee44-4687-9d30-e28f0be7a279" (UID: "a060e899-ee44-4687-9d30-e28f0be7a279"). InnerVolumeSpecName "kube-api-access-p7znj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:27.588374 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.588349    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-kube-api-access-82ntr" (OuterVolumeSpecName: "kube-api-access-82ntr") pod "4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" (UID: "4850a56b-3199-4b8a-af1d-1c1e4e3d5b23"). InnerVolumeSpecName "kube-api-access-82ntr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:27.688457 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.688374    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82ntr\" (UniqueName: \"kubernetes.io/projected/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23-kube-api-access-82ntr\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.688457 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.688406    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7znj\" (UniqueName: \"kubernetes.io/projected/a060e899-ee44-4687-9d30-e28f0be7a279-kube-api-access-p7znj\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:27.944504 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.944414    2568 generic.go:356] "Generic (PLEG): container finished" podID="4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" containerID="253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd" exitCode=0
Sep 18 19:06:27.944504 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.944466    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"
Sep 18 19:06:27.944716 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.944506    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x" event={"ID":"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23","Type":"ContainerDied","Data":"253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd"}
Sep 18 19:06:27.944716 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.944623    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x" event={"ID":"4850a56b-3199-4b8a-af1d-1c1e4e3d5b23","Type":"ContainerDied","Data":"6ff8329cbfa70800852915170f368b0a071541f33347b2d0220d95759ab77de1"}
Sep 18 19:06:27.944716 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.944651    2568 scope.go:117] "RemoveContainer" containerID="253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd"
Sep 18 19:06:27.946030 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.946000    2568 generic.go:356] "Generic (PLEG): container finished" podID="a060e899-ee44-4687-9d30-e28f0be7a279" containerID="93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990" exitCode=0
Sep 18 19:06:27.946162 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.946039    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"
Sep 18 19:06:27.946162 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.946040    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k" event={"ID":"a060e899-ee44-4687-9d30-e28f0be7a279","Type":"ContainerDied","Data":"93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990"}
Sep 18 19:06:27.946162 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.946084    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k" event={"ID":"a060e899-ee44-4687-9d30-e28f0be7a279","Type":"ContainerDied","Data":"3b6926f3ae537aa5cc3a343bfe004608a8dcf22d71c333eab9025bc240a844f9"}
Sep 18 19:06:27.954130 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.954110    2568 scope.go:117] "RemoveContainer" containerID="253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd"
Sep 18 19:06:27.954412 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:06:27.954388    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd\": container with ID starting with 253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd not found: ID does not exist" containerID="253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd"
Sep 18 19:06:27.954490 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.954423    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd"} err="failed to get container status \"253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd\": rpc error: code = NotFound desc = could not find container \"253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd\": container with ID starting with 253bcb9f5ebdfe1ebdb1b4f6087ddee2f59aec614fcee12921e9ba9ed03b74fd not found: ID does not exist"
Sep 18 19:06:27.954490 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.954446    2568 scope.go:117] "RemoveContainer" containerID="93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990"
Sep 18 19:06:27.963605 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.963588    2568 scope.go:117] "RemoveContainer" containerID="93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990"
Sep 18 19:06:27.963851 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:06:27.963832    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990\": container with ID starting with 93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990 not found: ID does not exist" containerID="93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990"
Sep 18 19:06:27.963898 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.963859    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990"} err="failed to get container status \"93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990\": rpc error: code = NotFound desc = could not find container \"93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990\": container with ID starting with 93335b50e8bf04b6bc4b5ee56caf3b9377f93fb249b96d70c81afa42cfc89990 not found: ID does not exist"
Sep 18 19:06:27.981948 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.981922    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"]
Sep 18 19:06:27.993376 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:27.993337    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-15c3ed-bd96fd686-r2b4x"]
Sep 18 19:06:28.017833 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:28.017808    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"]
Sep 18 19:06:28.020281 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:28.020262    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-15c3ed-759c7bb6b7-rh99k"]
Sep 18 19:06:29.073536 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:29.068713    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" path="/var/lib/kubelet/pods/4850a56b-3199-4b8a-af1d-1c1e4e3d5b23/volumes"
Sep 18 19:06:29.073536 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:29.069164    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73464364-b803-4183-a5fa-d2e0b68e4b64" path="/var/lib/kubelet/pods/73464364-b803-4183-a5fa-d2e0b68e4b64/volumes"
Sep 18 19:06:29.073536 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:29.069588    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a060e899-ee44-4687-9d30-e28f0be7a279" path="/var/lib/kubelet/pods/a060e899-ee44-4687-9d30-e28f0be7a279/volumes"
Sep 18 19:06:29.073536 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:29.070026    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc3e2803-05a3-4563-a6d5-ab8f411bf9ef" path="/var/lib/kubelet/pods/dc3e2803-05a3-4563-a6d5-ab8f411bf9ef/volumes"
Sep 18 19:06:31.121754 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:31.121722    2568 scope.go:117] "RemoveContainer" containerID="e788d7aa2ed3f07cd0cc105f05538d0e632f4683ad593d580973d4fb2d5e08a2"
Sep 18 19:06:31.133423 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:31.133399    2568 scope.go:117] "RemoveContainer" containerID="15fa15a049a8163c28384871f315a9c3980a70262be5cb79cec4291f57b18546"
Sep 18 19:06:31.142074 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:31.142056    2568 scope.go:117] "RemoveContainer" containerID="6a251e02b935bdf09f1fc199f338d49155ba0e633019e70444561723d87af903"
Sep 18 19:06:31.150361 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:31.150340    2568 scope.go:117] "RemoveContainer" containerID="041d79e705ecfb23eda5f9834e90a8d0f8cb9a3afaf6887d428268059e1028af"
Sep 18 19:06:38.683738 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:38.683705    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"]
Sep 18 19:06:38.684299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:38.683954    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy" containerID="cri-o://4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32" gracePeriod=30
Sep 18 19:06:39.152769 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.152736    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"]
Sep 18 19:06:39.153172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153161    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a060e899-ee44-4687-9d30-e28f0be7a279" containerName="payload-processing"
Sep 18 19:06:39.153217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153175    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a060e899-ee44-4687-9d30-e28f0be7a279" containerName="payload-processing"
Sep 18 19:06:39.153217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153184    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc3e2803-05a3-4563-a6d5-ab8f411bf9ef" containerName="revoke-keys"
Sep 18 19:06:39.153217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153191    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc3e2803-05a3-4563-a6d5-ab8f411bf9ef" containerName="revoke-keys"
Sep 18 19:06:39.153311 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153219    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" containerName="payload-pre-processing"
Sep 18 19:06:39.153311 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153225    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" containerName="payload-pre-processing"
Sep 18 19:06:39.153376 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153311    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc3e2803-05a3-4563-a6d5-ab8f411bf9ef" containerName="revoke-keys"
Sep 18 19:06:39.153376 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153323    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4850a56b-3199-4b8a-af1d-1c1e4e3d5b23" containerName="payload-pre-processing"
Sep 18 19:06:39.153376 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.153332    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a060e899-ee44-4687-9d30-e28f0be7a279" containerName="payload-processing"
Sep 18 19:06:39.158537 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.158498    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:39.162947 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.162928    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:06:39.163823 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.163804    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:06:39.178760 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.178731    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"]
Sep 18 19:06:39.316819 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.316788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d3be10-3437-4253-97e4-c80c0d65b8b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9\" (UID: \"41d3be10-3437-4253-97e4-c80c0d65b8b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:39.417940 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.417856    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d3be10-3437-4253-97e4-c80c0d65b8b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9\" (UID: \"41d3be10-3437-4253-97e4-c80c0d65b8b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:39.418486 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.418468    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d3be10-3437-4253-97e4-c80c0d65b8b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9\" (UID: \"41d3be10-3437-4253-97e4-c80c0d65b8b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:39.468802 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.468765    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:39.768635 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:39.768607    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"]
Sep 18 19:06:39.770921 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:06:39.770893    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d3be10_3437_4253_97e4_c80c0d65b8b6.slice/crio-0d5658f5ec7320695caf2f7ff7ef882cf9373d0131e3ce3cc0d5c9b6c98ddec9 WatchSource:0}: Error finding container 0d5658f5ec7320695caf2f7ff7ef882cf9373d0131e3ce3cc0d5c9b6c98ddec9: Status 404 returned error can't find the container with id 0d5658f5ec7320695caf2f7ff7ef882cf9373d0131e3ce3cc0d5c9b6c98ddec9
Sep 18 19:06:40.020646 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:40.020562    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9" event={"ID":"41d3be10-3437-4253-97e4-c80c0d65b8b6","Type":"ContainerStarted","Data":"0d5658f5ec7320695caf2f7ff7ef882cf9373d0131e3ce3cc0d5c9b6c98ddec9"}
Sep 18 19:06:41.038851 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:41.038814    2568 generic.go:356] "Generic (PLEG): container finished" podID="41d3be10-3437-4253-97e4-c80c0d65b8b6" containerID="b86812635f8cfd556b91ff95607863f3a25c4efb8d993555e6cac9346934ffdb" exitCode=0
Sep 18 19:06:41.039309 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:41.038900    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9" event={"ID":"41d3be10-3437-4253-97e4-c80c0d65b8b6","Type":"ContainerDied","Data":"b86812635f8cfd556b91ff95607863f3a25c4efb8d993555e6cac9346934ffdb"}
Sep 18 19:06:42.194727 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:42.194697    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:42.349569 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:42.349447    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d3be10-3437-4253-97e4-c80c0d65b8b6-maas-api-service-ca\") pod \"41d3be10-3437-4253-97e4-c80c0d65b8b6\" (UID: \"41d3be10-3437-4253-97e4-c80c0d65b8b6\") "
Sep 18 19:06:42.349848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:42.349823    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41d3be10-3437-4253-97e4-c80c0d65b8b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41d3be10-3437-4253-97e4-c80c0d65b8b6" (UID: "41d3be10-3437-4253-97e4-c80c0d65b8b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:42.349930 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:42.349916    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d3be10-3437-4253-97e4-c80c0d65b8b6-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:43.056728 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.056697    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"
Sep 18 19:06:43.057409 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.057384    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9" event={"ID":"41d3be10-3437-4253-97e4-c80c0d65b8b6","Type":"ContainerDied","Data":"0d5658f5ec7320695caf2f7ff7ef882cf9373d0131e3ce3cc0d5c9b6c98ddec9"}
Sep 18 19:06:43.057479 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.057430    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d5658f5ec7320695caf2f7ff7ef882cf9373d0131e3ce3cc0d5c9b6c98ddec9"
Sep 18 19:06:43.582529 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.582485    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"]
Sep 18 19:06:43.583011 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.582996    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41d3be10-3437-4253-97e4-c80c0d65b8b6" containerName="revoke-keys"
Sep 18 19:06:43.583055 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.583013    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41d3be10-3437-4253-97e4-c80c0d65b8b6" containerName="revoke-keys"
Sep 18 19:06:43.583090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.583082    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="41d3be10-3437-4253-97e4-c80c0d65b8b6" containerName="revoke-keys"
Sep 18 19:06:43.586548 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.586529    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.590179 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.590153    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4ac9ac7d-dockercfg-q9drx\""
Sep 18 19:06:43.590367 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.590197    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4ac9ac7d\""
Sep 18 19:06:43.600243 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.600209    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"]
Sep 18 19:06:43.666364 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.666325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6fs\" (UniqueName: \"kubernetes.io/projected/2918dfe3-682e-4f11-887d-e31309836e9f-kube-api-access-tx6fs\") pod \"payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") " pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.666594 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.666485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2918dfe3-682e-4f11-887d-e31309836e9f-plugins-config-volume\") pod \"payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") " pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.767971 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.767937    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2918dfe3-682e-4f11-887d-e31309836e9f-plugins-config-volume\") pod \"payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") " pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.768366 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.768333    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6fs\" (UniqueName: \"kubernetes.io/projected/2918dfe3-682e-4f11-887d-e31309836e9f-kube-api-access-tx6fs\") pod \"payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") " pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.768909 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.768859    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2918dfe3-682e-4f11-887d-e31309836e9f-plugins-config-volume\") pod \"payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") " pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.778233 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.778188    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6fs\" (UniqueName: \"kubernetes.io/projected/2918dfe3-682e-4f11-887d-e31309836e9f-kube-api-access-tx6fs\") pod \"payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") " pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.898300 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.898264    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:43.936218 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:43.936194    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 19:06:44.064053 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.064019    2568 generic.go:356] "Generic (PLEG): container finished" podID="c12a16a9-f448-4310-bf6e-c82746b28283" containerID="4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32" exitCode=0
Sep 18 19:06:44.064246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.064061    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"
Sep 18 19:06:44.064246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.064108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" event={"ID":"c12a16a9-f448-4310-bf6e-c82746b28283","Type":"ContainerDied","Data":"4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32"}
Sep 18 19:06:44.064246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.064168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv" event={"ID":"c12a16a9-f448-4310-bf6e-c82746b28283","Type":"ContainerDied","Data":"588ac6d981ece04d8198139ada1ba8bfef739a66ccfe7f590a6c44325264238a"}
Sep 18 19:06:44.064246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.064198    2568 scope.go:117] "RemoveContainer" containerID="4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32"
Sep 18 19:06:44.070946 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.070926    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-data\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071050 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.070978    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-istio-token\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071050 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071000    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-envoy\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071050 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071029    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-credential-socket\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071191 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071061    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-socket\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071191 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071108    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12a16a9-f448-4310-bf6e-c82746b28283-istiod-ca-cert\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071191 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071188    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-certs\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071353 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071216    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-499q7\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-kube-api-access-499q7\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071353 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071244    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-data" (OuterVolumeSpecName: "istio-data") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:44.071353 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071257    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12a16a9-f448-4310-bf6e-c82746b28283-istio-podinfo\") pod \"c12a16a9-f448-4310-bf6e-c82746b28283\" (UID: \"c12a16a9-f448-4310-bf6e-c82746b28283\") "
Sep 18 19:06:44.071507 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071426    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:44.071596 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071498    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:44.071849 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071826    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.071923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071854    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.071923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071868    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.071923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.071883    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:44.072196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.072170    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c12a16a9-f448-4310-bf6e-c82746b28283-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:06:44.073411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.073379    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c12a16a9-f448-4310-bf6e-c82746b28283-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:06:44.073411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.073400    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-istio-token" (OuterVolumeSpecName: "istio-token") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:44.073813 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.073781    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-kube-api-access-499q7" (OuterVolumeSpecName: "kube-api-access-499q7") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "kube-api-access-499q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:06:44.073813 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.073787    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c12a16a9-f448-4310-bf6e-c82746b28283" (UID: "c12a16a9-f448-4310-bf6e-c82746b28283"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:06:44.074190 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.074170    2568 scope.go:117] "RemoveContainer" containerID="9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"
Sep 18 19:06:44.088186 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.088168    2568 scope.go:117] "RemoveContainer" containerID="4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32"
Sep 18 19:06:44.088444 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:06:44.088424    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32\": container with ID starting with 4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32 not found: ID does not exist" containerID="4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32"
Sep 18 19:06:44.088486 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.088454    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32"} err="failed to get container status \"4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32\": rpc error: code = NotFound desc = could not find container \"4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32\": container with ID starting with 4a8cbae35bf63307deb5bcc5ec058340a4c266c2c8c5fb7ba302501501a92f32 not found: ID does not exist"
Sep 18 19:06:44.088486 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.088471    2568 scope.go:117] "RemoveContainer" containerID="9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"
Sep 18 19:06:44.088779 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:06:44.088760    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6\": container with ID starting with 9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6 not found: ID does not exist" containerID="9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"
Sep 18 19:06:44.088833 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.088784    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6"} err="failed to get container status \"9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6\": rpc error: code = NotFound desc = could not find container \"9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6\": container with ID starting with 9045aec71cc30dcbc9232909fcec0e9f7f69a053012cf4a793146e725e6180a6 not found: ID does not exist"
Sep 18 19:06:44.173411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.173329    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.173411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.173353    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.173411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.173362    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12a16a9-f448-4310-bf6e-c82746b28283-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.173411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.173370    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12a16a9-f448-4310-bf6e-c82746b28283-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.173411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.173379    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-499q7\" (UniqueName: \"kubernetes.io/projected/c12a16a9-f448-4310-bf6e-c82746b28283-kube-api-access-499q7\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.173411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.173388    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12a16a9-f448-4310-bf6e-c82746b28283-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:06:44.178596 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.178572    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"]
Sep 18 19:06:44.179060 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:06:44.179033    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2918dfe3_682e_4f11_887d_e31309836e9f.slice/crio-f9079feca336600c3b629754f42a3aa1a010b1712d12c60e9819717c6a6e128e WatchSource:0}: Error finding container f9079feca336600c3b629754f42a3aa1a010b1712d12c60e9819717c6a6e128e: Status 404 returned error can't find the container with id f9079feca336600c3b629754f42a3aa1a010b1712d12c60e9819717c6a6e128e
Sep 18 19:06:44.390408 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.390378    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"]
Sep 18 19:06:44.399201 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:44.399171    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-15c3ed-openshift-default-c647c46dd-c5rkv"]
Sep 18 19:06:45.055918 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:45.055886    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" path="/var/lib/kubelet/pods/c12a16a9-f448-4310-bf6e-c82746b28283/volumes"
Sep 18 19:06:45.071132 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:45.071100    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t" event={"ID":"2918dfe3-682e-4f11-887d-e31309836e9f","Type":"ContainerStarted","Data":"4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8"}
Sep 18 19:06:45.071334 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:45.071154    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t" event={"ID":"2918dfe3-682e-4f11-887d-e31309836e9f","Type":"ContainerStarted","Data":"f9079feca336600c3b629754f42a3aa1a010b1712d12c60e9819717c6a6e128e"}
Sep 18 19:06:45.071334 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:45.071203    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:06:45.102608 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:45.102543    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t" podStartSLOduration=2.102510192 podStartE2EDuration="2.102510192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:06:45.102165219 +0000 UTC m=+1334.540425430" watchObservedRunningTime="2026-09-18 19:06:45.102510192 +0000 UTC m=+1334.540770396"
Sep 18 19:06:49.698495 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:49.698465    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"]
Sep 18 19:06:49.704170 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:49.704143    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-3a81e5-29829300-mnwqn"]
Sep 18 19:06:50.013903 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:50.013871    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"]
Sep 18 19:06:50.033267 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:50.033235    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3a81e5-slgz9"]
Sep 18 19:06:51.056296 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:51.056261    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41d3be10-3437-4253-97e4-c80c0d65b8b6" path="/var/lib/kubelet/pods/41d3be10-3437-4253-97e4-c80c0d65b8b6/volumes"
Sep 18 19:06:51.056822 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:51.056741    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1be9f1-6611-4421-a5ab-96dbc0d13bee" path="/var/lib/kubelet/pods/cc1be9f1-6611-4421-a5ab-96dbc0d13bee/volumes"
Sep 18 19:06:56.080743 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:06:56.080661    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:07:04.841550 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:04.841501    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"]
Sep 18 19:07:04.842000 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:04.841755    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49" podUID="dc69e4b0-57df-4ecb-a224-daddfbe42efd" containerName="istio-proxy" containerID="cri-o://30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d" gracePeriod=30
Sep 18 19:07:10.113600 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.113533    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:07:10.220842 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.220809    2568 generic.go:356] "Generic (PLEG): container finished" podID="dc69e4b0-57df-4ecb-a224-daddfbe42efd" containerID="30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d" exitCode=0
Sep 18 19:07:10.221045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.220871    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"
Sep 18 19:07:10.221045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.220889    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49" event={"ID":"dc69e4b0-57df-4ecb-a224-daddfbe42efd","Type":"ContainerDied","Data":"30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d"}
Sep 18 19:07:10.221045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.220943    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49" event={"ID":"dc69e4b0-57df-4ecb-a224-daddfbe42efd","Type":"ContainerDied","Data":"4d7100823ad1dd799f126cde8edbfc5040029e13123ff81e31db4808e6d8b413"}
Sep 18 19:07:10.221045 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.220972    2568 scope.go:117] "RemoveContainer" containerID="30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d"
Sep 18 19:07:10.230423 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.230403    2568 scope.go:117] "RemoveContainer" containerID="30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d"
Sep 18 19:07:10.230824 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:07:10.230790    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d\": container with ID starting with 30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d not found: ID does not exist" containerID="30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d"
Sep 18 19:07:10.230917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.230836    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d"} err="failed to get container status \"30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d\": rpc error: code = NotFound desc = could not find container \"30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d\": container with ID starting with 30f851ca6561f442947b7e310719626c11514476e6c6da1f953c3d2eb3f5682d not found: ID does not exist"
Sep 18 19:07:10.239125 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239100    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-envoy\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239280 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239142    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-data\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239280 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239159    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-credential-socket\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239280 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239192    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-token\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239280 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239241    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-podinfo\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239342    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-socket\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239376    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-certs\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239403    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58cmd\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-kube-api-access-58cmd\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239441    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istiod-ca-cert\") pod \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\" (UID: \"dc69e4b0-57df-4ecb-a224-daddfbe42efd\") "
Sep 18 19:07:10.239493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239451    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:10.239809 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239536    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:10.239809 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239655    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:10.239952 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239915    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.240010 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239959    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.240010 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239958    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:10.240010 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.239973    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.240010 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.240002    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:07:10.241621 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.241585    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:10.241907 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.241885    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-kube-api-access-58cmd" (OuterVolumeSpecName: "kube-api-access-58cmd") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "kube-api-access-58cmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:07:10.241985 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.241961    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:07:10.242046 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.241977    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc69e4b0-57df-4ecb-a224-daddfbe42efd" (UID: "dc69e4b0-57df-4ecb-a224-daddfbe42efd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:07:10.340883 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.340845    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.340883 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.340878    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.341086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.340893    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.341086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.340906    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58cmd\" (UniqueName: \"kubernetes.io/projected/dc69e4b0-57df-4ecb-a224-daddfbe42efd-kube-api-access-58cmd\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.341086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.340922    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.341086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.340935    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc69e4b0-57df-4ecb-a224-daddfbe42efd-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:10.565271 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.565231    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"]
Sep 18 19:07:10.581815 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:10.581778    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-9674d9-openshift-default-5bcb885b7d-5vt49"]
Sep 18 19:07:11.054923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:11.054891    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc69e4b0-57df-4ecb-a224-daddfbe42efd" path="/var/lib/kubelet/pods/dc69e4b0-57df-4ecb-a224-daddfbe42efd/volumes"
Sep 18 19:07:31.191061 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:31.191032    2568 scope.go:117] "RemoveContainer" containerID="6ee4612ff41bbb9562ae5ee6a9d97678dacb1b932610266760c990e4a6ad30f2"
Sep 18 19:07:31.199656 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:31.199638    2568 scope.go:117] "RemoveContainer" containerID="04d2de4c4fa8b3b047da95a46a111fcf0cfe6a03c0cafcc6bd6304fbf1b128ce"
Sep 18 19:07:31.207676 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:31.207651    2568 scope.go:117] "RemoveContainer" containerID="59610dea0d40b97c62debfa432df742e4460837b40f2027cff7558e246f3b4df"
Sep 18 19:07:34.179195 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:34.179162    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"]
Sep 18 19:07:34.179629 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:34.179416    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" containerID="cri-o://e5fa712e41c0678341bab2deb458f675a9545a032d95c9fece9c17283cde9a80" gracePeriod=30
Sep 18 19:07:35.114206 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114169    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"]
Sep 18 19:07:35.114912 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114886    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc69e4b0-57df-4ecb-a224-daddfbe42efd" containerName="istio-proxy"
Sep 18 19:07:35.114912 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114914    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc69e4b0-57df-4ecb-a224-daddfbe42efd" containerName="istio-proxy"
Sep 18 19:07:35.115054 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114946    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy"
Sep 18 19:07:35.115054 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114956    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy"
Sep 18 19:07:35.115054 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114974    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy"
Sep 18 19:07:35.115054 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.114983    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy"
Sep 18 19:07:35.115181 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.115102    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc69e4b0-57df-4ecb-a224-daddfbe42efd" containerName="istio-proxy"
Sep 18 19:07:35.115181 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.115118    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy"
Sep 18 19:07:35.115181 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.115132    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12a16a9-f448-4310-bf6e-c82746b28283" containerName="istio-proxy"
Sep 18 19:07:35.120325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.120306    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.124325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.124303    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-feb80d-openshift-default-dockercfg-f8dh6\""
Sep 18 19:07:35.135576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.135550    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"]
Sep 18 19:07:35.194971 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.194940    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-token\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.194992    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-data\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195041    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-credential-socket\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-podinfo\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-certs\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195198    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hfqj\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-kube-api-access-8hfqj\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-envoy\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195310    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-socket\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.195336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.195325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c76fd339-cc47-4f20-9997-6ae985bc4d03-istiod-ca-cert\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296462    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hfqj\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-kube-api-access-8hfqj\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296691 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296552    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-envoy\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296691 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296582    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-socket\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296691 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296598    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c76fd339-cc47-4f20-9997-6ae985bc4d03-istiod-ca-cert\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296862 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296766    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-token\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296921 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-data\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.296921 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296899    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-credential-socket\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297024 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.296939    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-podinfo\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297024 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.297015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-certs\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297360 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.297339    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c76fd339-cc47-4f20-9997-6ae985bc4d03-istiod-ca-cert\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297360 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.297348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-credential-socket\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297501 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.297366    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-certs\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297501 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.297475    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-data\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.297647 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.297475    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-socket\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.299389 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.299367    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-envoy\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.299637 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.299616    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-podinfo\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.304822 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.304800    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-token\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.305148 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.305132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hfqj\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-kube-api-access-8hfqj\") pod \"e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") " pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.431396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.431310    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:35.712939 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.709582    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"]
Sep 18 19:07:35.714947 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.714905    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:07:35.715036 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.714996    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:07:35.715075 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:35.715033    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:07:36.386533 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:36.386475    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" event={"ID":"c76fd339-cc47-4f20-9997-6ae985bc4d03","Type":"ContainerStarted","Data":"c2f5349beed3e24746a0ce62b7120ea3637f495a459ae33bc233de5a4abf10db"}
Sep 18 19:07:36.387034 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:36.386552    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" event={"ID":"c76fd339-cc47-4f20-9997-6ae985bc4d03","Type":"ContainerStarted","Data":"c82840e146e6b17c6676b583b0422b5f75ba8a8280c6b0c69251b98969e1be12"}
Sep 18 19:07:36.420818 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:36.420771    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podStartSLOduration=1.420754336 podStartE2EDuration="1.420754336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:07:36.418126995 +0000 UTC m=+1385.856387198" watchObservedRunningTime="2026-09-18 19:07:36.420754336 +0000 UTC m=+1385.859014539"
Sep 18 19:07:36.431690 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:36.431656    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:36.437216 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:36.437193    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:37.392384 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:37.392353    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:37.393450 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:37.393430    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:07:39.198977 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.198939    2568 patch_prober.go:28] interesting pod/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 18 19:07:39.199290 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.198998    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 18 19:07:39.407077 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.407038    2568 generic.go:356] "Generic (PLEG): container finished" podID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerID="e5fa712e41c0678341bab2deb458f675a9545a032d95c9fece9c17283cde9a80" exitCode=0
Sep 18 19:07:39.407291 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.407121    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" event={"ID":"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67","Type":"ContainerDied","Data":"e5fa712e41c0678341bab2deb458f675a9545a032d95c9fece9c17283cde9a80"}
Sep 18 19:07:39.407291 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.407199    2568 scope.go:117] "RemoveContainer" containerID="2208783ba84dafbd176bd7831a999b3f179727dba3a9bb735cb30d8235a48515"
Sep 18 19:07:39.448708 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.448643    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 19:07:39.541616 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541582    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-envoy\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541616 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541614    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-data\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541651    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istiod-ca-cert\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541689    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-socket\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541723    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-podinfo\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541756    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn2kw\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-kube-api-access-vn2kw\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541790    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-credential-socket\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.541863 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541830    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-token\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.542172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541868    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-certs\") pod \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\" (UID: \"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67\") "
Sep 18 19:07:39.542172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.541911    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-data" (OuterVolumeSpecName: "istio-data") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:39.542172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542092    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:39.542172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542152    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:39.542387 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542190    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:07:39.542387 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542294    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:39.542474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542393    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.542474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542417    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.542474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542434    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.542474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542448    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.542474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.542461    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.543928 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.543907    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-kube-api-access-vn2kw" (OuterVolumeSpecName: "kube-api-access-vn2kw") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "kube-api-access-vn2kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:07:39.544057 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.544041    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:07:39.544358 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.544342    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-token" (OuterVolumeSpecName: "istio-token") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:07:39.544440 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.544353    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" (UID: "7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:07:39.643501 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.643464    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.643501 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.643494    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.643501 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.643502    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:39.643754 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:39.643511    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn2kw\" (UniqueName: \"kubernetes.io/projected/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67-kube-api-access-vn2kw\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:07:40.414488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:40.414457    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"
Sep 18 19:07:40.414970 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:40.414481    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8" event={"ID":"7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67","Type":"ContainerDied","Data":"3a98b6509fb64c0d9ea5572beae67e09e53db3c1154f8394cb7d57169fd87aa4"}
Sep 18 19:07:40.414970 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:40.414582    2568 scope.go:117] "RemoveContainer" containerID="e5fa712e41c0678341bab2deb458f675a9545a032d95c9fece9c17283cde9a80"
Sep 18 19:07:40.439463 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:40.439429    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"]
Sep 18 19:07:40.446071 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:40.446047    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3a81e5-openshift-default-8d7457b9d-qkbb8"]
Sep 18 19:07:41.055829 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:41.055787    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" path="/var/lib/kubelet/pods/7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67/volumes"
Sep 18 19:07:44.496968 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.496935    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"]
Sep 18 19:07:44.497407 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.497382    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy"
Sep 18 19:07:44.497407 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.497403    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy"
Sep 18 19:07:44.497498 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.497424    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy"
Sep 18 19:07:44.497498 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.497432    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy"
Sep 18 19:07:44.497595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.497542    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy"
Sep 18 19:07:44.497595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.497558    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4ebd3-e89f-46f9-bcb2-861ca1cbdb67" containerName="istio-proxy"
Sep 18 19:07:44.502596 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.502578    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.505738 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.505719    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-feb80d-dockercfg-bljml\""
Sep 18 19:07:44.506058 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.506041    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-feb80d\""
Sep 18 19:07:44.517236 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.517212    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"]
Sep 18 19:07:44.553615 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.553587    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"]
Sep 18 19:07:44.557408 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.557390    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.569961 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.569936    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"]
Sep 18 19:07:44.593447 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.593414    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42bt2\" (UniqueName: \"kubernetes.io/projected/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-kube-api-access-42bt2\") pod \"payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.593654 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.593568    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.694092 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.694058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8pfp\" (UniqueName: \"kubernetes.io/projected/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-kube-api-access-f8pfp\") pod \"payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.694283 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.694118    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42bt2\" (UniqueName: \"kubernetes.io/projected/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-kube-api-access-42bt2\") pod \"payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.694283 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.694200    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.694418 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.694302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.694966 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.694949    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.707859 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.707832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42bt2\" (UniqueName: \"kubernetes.io/projected/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-kube-api-access-42bt2\") pod \"payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.795490 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.795403    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.795676 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.795541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8pfp\" (UniqueName: \"kubernetes.io/projected/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-kube-api-access-f8pfp\") pod \"payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.796123 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.796102    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.803090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.803065    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8pfp\" (UniqueName: \"kubernetes.io/projected/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-kube-api-access-f8pfp\") pod \"payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:44.812928 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.812906    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:44.867785 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:44.867756    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:45.086002 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.085972    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"]
Sep 18 19:07:45.087558 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:07:45.087529    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca3301d4_6b12_4192_b8c7_b1c93372b0a1.slice/crio-40109a440e092d08fe7e832e878ce2ed359c0096ae939cdc878a50542276fc1b WatchSource:0}: Error finding container 40109a440e092d08fe7e832e878ce2ed359c0096ae939cdc878a50542276fc1b: Status 404 returned error can't find the container with id 40109a440e092d08fe7e832e878ce2ed359c0096ae939cdc878a50542276fc1b
Sep 18 19:07:45.152295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.152267    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"]
Sep 18 19:07:45.153667 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:07:45.153644    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc9fd3e_4b80_4f56_ba51_36e8a8c85bb7.slice/crio-fb71bb7ede8dfe0e1df70b0e9aa33b9eb48f0227db2c4fafd449064b5389d690 WatchSource:0}: Error finding container fb71bb7ede8dfe0e1df70b0e9aa33b9eb48f0227db2c4fafd449064b5389d690: Status 404 returned error can't find the container with id fb71bb7ede8dfe0e1df70b0e9aa33b9eb48f0227db2c4fafd449064b5389d690
Sep 18 19:07:45.446395 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.446286    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh" event={"ID":"ca3301d4-6b12-4192-b8c7-b1c93372b0a1","Type":"ContainerStarted","Data":"e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607"}
Sep 18 19:07:45.446395 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.446357    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:45.446395 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.446387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh" event={"ID":"ca3301d4-6b12-4192-b8c7-b1c93372b0a1","Type":"ContainerStarted","Data":"40109a440e092d08fe7e832e878ce2ed359c0096ae939cdc878a50542276fc1b"}
Sep 18 19:07:45.448133 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.448103    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc" event={"ID":"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7","Type":"ContainerStarted","Data":"e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964"}
Sep 18 19:07:45.448257 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.448159    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc" event={"ID":"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7","Type":"ContainerStarted","Data":"fb71bb7ede8dfe0e1df70b0e9aa33b9eb48f0227db2c4fafd449064b5389d690"}
Sep 18 19:07:45.448257 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.448188    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:07:45.473575 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.473506    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh" podStartSLOduration=1.473487702 podStartE2EDuration="1.473487702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:07:45.470707489 +0000 UTC m=+1394.908967694" watchObservedRunningTime="2026-09-18 19:07:45.473487702 +0000 UTC m=+1394.911747906"
Sep 18 19:07:45.507394 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:45.507346    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc" podStartSLOduration=1.507329372 podStartE2EDuration="1.507329372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:07:45.50608442 +0000 UTC m=+1394.944344624" watchObservedRunningTime="2026-09-18 19:07:45.507329372 +0000 UTC m=+1394.945589575"
Sep 18 19:07:56.456872 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:56.456841    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:07:56.457250 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:07:56.456998    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:08:04.567581 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:04.567547    2568 generic.go:356] "Generic (PLEG): container finished" podID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerID="c2f5349beed3e24746a0ce62b7120ea3637f495a459ae33bc233de5a4abf10db" exitCode=0
Sep 18 19:08:04.568046 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:04.567624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" event={"ID":"c76fd339-cc47-4f20-9997-6ae985bc4d03","Type":"ContainerDied","Data":"c2f5349beed3e24746a0ce62b7120ea3637f495a459ae33bc233de5a4abf10db"}
Sep 18 19:08:04.568046 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:04.568030    2568 scope.go:117] "RemoveContainer" containerID="c2f5349beed3e24746a0ce62b7120ea3637f495a459ae33bc233de5a4abf10db"
Sep 18 19:08:04.569169 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:04.569139    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:08:04.569253 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:04.569204    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:08:04.569253 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:04.569238    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:08:05.575122 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:05.575077    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" event={"ID":"c76fd339-cc47-4f20-9997-6ae985bc4d03","Type":"ContainerStarted","Data":"b32e84103188a891a017a85d217bf5e6d35b00bff6fb3d8468ddaff19b6f7f40"}
Sep 18 19:08:06.432081 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:06.432045    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:08:06.433753 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:06.433723    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:08:06.434008 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:06.433977    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 19:08:07.392688 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:07.392658    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:08:07.433941 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:07.433476    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:08:07.433941 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:07.433562    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 19:08:08.432387 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:08.432342    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:08:08.432961 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:08.432414    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 19:08:09.432257 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:09.431743    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:08:09.432257 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:09.431842    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 19:08:10.432620 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:10.432579    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 19:08:10.433230 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:10.433193    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 19:08:11.436688 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:11.436655    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:08:11.437758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:11.437739    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:08:16.097317 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.097277    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"]
Sep 18 19:08:16.100753 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.100734    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.103296 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.103273    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-19944d\""
Sep 18 19:08:16.103549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.103492    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-19944d-dockercfg-x7cpd\""
Sep 18 19:08:16.128449 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.128422    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"]
Sep 18 19:08:16.130859 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.130835    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"]
Sep 18 19:08:16.133362 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.133341    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.143463 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.143434    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68a0165-d103-46b9-aee1-0e8cfd099e66-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.143737 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.143674    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nddg\" (UniqueName: \"kubernetes.io/projected/d68a0165-d103-46b9-aee1-0e8cfd099e66-kube-api-access-8nddg\") pod \"payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.147508 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.147482    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"]
Sep 18 19:08:16.245285 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.245247    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.245444 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.245290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nddg\" (UniqueName: \"kubernetes.io/projected/d68a0165-d103-46b9-aee1-0e8cfd099e66-kube-api-access-8nddg\") pod \"payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.245444 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.245341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qktz\" (UniqueName: \"kubernetes.io/projected/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-kube-api-access-5qktz\") pod \"payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.245550 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.245496    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68a0165-d103-46b9-aee1-0e8cfd099e66-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.246152 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.246134    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68a0165-d103-46b9-aee1-0e8cfd099e66-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.254710 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.254687    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nddg\" (UniqueName: \"kubernetes.io/projected/d68a0165-d103-46b9-aee1-0e8cfd099e66-kube-api-access-8nddg\") pod \"payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.347132 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.346975    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.347367 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.347161    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qktz\" (UniqueName: \"kubernetes.io/projected/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-kube-api-access-5qktz\") pod \"payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.347901 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.347873    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.358962 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.358920    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qktz\" (UniqueName: \"kubernetes.io/projected/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-kube-api-access-5qktz\") pod \"payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.411818 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.411778    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:16.446750 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.446717    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:16.698313 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.698289    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"]
Sep 18 19:08:16.700122 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:08:16.700099    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68a0165_d103_46b9_aee1_0e8cfd099e66.slice/crio-4549f903cb0021e8af8a9a8906eb05c67ff167a13b1f256285100185377cf15c WatchSource:0}: Error finding container 4549f903cb0021e8af8a9a8906eb05c67ff167a13b1f256285100185377cf15c: Status 404 returned error can't find the container with id 4549f903cb0021e8af8a9a8906eb05c67ff167a13b1f256285100185377cf15c
Sep 18 19:08:16.756297 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:08:16.755945    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ec8680_419d_4e3f_b1db_6e2fe9988ef1.slice/crio-392aa7af84d0b54767d2fcf847a4fbb5bae8a581d9bc3fdcfb8e14eaac5a2f6a WatchSource:0}: Error finding container 392aa7af84d0b54767d2fcf847a4fbb5bae8a581d9bc3fdcfb8e14eaac5a2f6a: Status 404 returned error can't find the container with id 392aa7af84d0b54767d2fcf847a4fbb5bae8a581d9bc3fdcfb8e14eaac5a2f6a
Sep 18 19:08:16.759368 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:16.759349    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"]
Sep 18 19:08:17.649121 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.649082    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd" event={"ID":"23ec8680-419d-4e3f-b1db-6e2fe9988ef1","Type":"ContainerStarted","Data":"d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a"}
Sep 18 19:08:17.649667 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.649152    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd" event={"ID":"23ec8680-419d-4e3f-b1db-6e2fe9988ef1","Type":"ContainerStarted","Data":"392aa7af84d0b54767d2fcf847a4fbb5bae8a581d9bc3fdcfb8e14eaac5a2f6a"}
Sep 18 19:08:17.649667 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.649332    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:17.650842 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.650820    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf" event={"ID":"d68a0165-d103-46b9-aee1-0e8cfd099e66","Type":"ContainerStarted","Data":"96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680"}
Sep 18 19:08:17.650955 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.650860    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf" event={"ID":"d68a0165-d103-46b9-aee1-0e8cfd099e66","Type":"ContainerStarted","Data":"4549f903cb0021e8af8a9a8906eb05c67ff167a13b1f256285100185377cf15c"}
Sep 18 19:08:17.650955 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.650902    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:17.679361 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.679307    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd" podStartSLOduration=1.679293089 podStartE2EDuration="1.679293089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:17.675051702 +0000 UTC m=+1427.113311905" watchObservedRunningTime="2026-09-18 19:08:17.679293089 +0000 UTC m=+1427.117553344"
Sep 18 19:08:17.709812 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:17.709766    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf" podStartSLOduration=1.7097514550000001 podStartE2EDuration="1.709751455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:08:17.706619937 +0000 UTC m=+1427.144880144" watchObservedRunningTime="2026-09-18 19:08:17.709751455 +0000 UTC m=+1427.148011659"
Sep 18 19:08:28.660496 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:28.660464    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:08:28.660969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:28.660721    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:08:31.242928 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:31.242897    2568 scope.go:117] "RemoveContainer" containerID="893f57e6be2c2965ff5ce6d8620ad0f7d3db71f7fea3fb36c7e657611dbd754c"
Sep 18 19:08:33.322633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.322589    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"]
Sep 18 19:08:33.328149 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.328123    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:33.330557 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.330537    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:08:33.331175 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.331155    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:08:33.341967 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.341946    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"]
Sep 18 19:08:33.430003 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.429964    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008158fc-b8ea-492f-b861-2ff1a09095d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6\" (UID: \"008158fc-b8ea-492f-b861-2ff1a09095d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:33.531571 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.531537    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008158fc-b8ea-492f-b861-2ff1a09095d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6\" (UID: \"008158fc-b8ea-492f-b861-2ff1a09095d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:33.532139 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.532119    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008158fc-b8ea-492f-b861-2ff1a09095d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6\" (UID: \"008158fc-b8ea-492f-b861-2ff1a09095d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:33.639539 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.639403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:33.924471 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:08:33.924439    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008158fc_b8ea_492f_b861_2ff1a09095d7.slice/crio-6a401d8cff7db9382b0bd16b395e665453d82ce2e5f483e9a4842961f0717846 WatchSource:0}: Error finding container 6a401d8cff7db9382b0bd16b395e665453d82ce2e5f483e9a4842961f0717846: Status 404 returned error can't find the container with id 6a401d8cff7db9382b0bd16b395e665453d82ce2e5f483e9a4842961f0717846
Sep 18 19:08:33.925394 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:33.925367    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"]
Sep 18 19:08:34.757961 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:34.757923    2568 generic.go:356] "Generic (PLEG): container finished" podID="008158fc-b8ea-492f-b861-2ff1a09095d7" containerID="9e6f2724ce8dc098c3bce6fc55cf094f1fe0e6f3a6cec8fc4707eccf878e1ae5" exitCode=0
Sep 18 19:08:34.758325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:34.757995    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6" event={"ID":"008158fc-b8ea-492f-b861-2ff1a09095d7","Type":"ContainerDied","Data":"9e6f2724ce8dc098c3bce6fc55cf094f1fe0e6f3a6cec8fc4707eccf878e1ae5"}
Sep 18 19:08:34.758325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:34.758033    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6" event={"ID":"008158fc-b8ea-492f-b861-2ff1a09095d7","Type":"ContainerStarted","Data":"6a401d8cff7db9382b0bd16b395e665453d82ce2e5f483e9a4842961f0717846"}
Sep 18 19:08:35.901021 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:35.900997    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:35.958441 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:35.958406    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008158fc-b8ea-492f-b861-2ff1a09095d7-maas-api-service-ca\") pod \"008158fc-b8ea-492f-b861-2ff1a09095d7\" (UID: \"008158fc-b8ea-492f-b861-2ff1a09095d7\") "
Sep 18 19:08:35.958745 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:35.958719    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/008158fc-b8ea-492f-b861-2ff1a09095d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "008158fc-b8ea-492f-b861-2ff1a09095d7" (UID: "008158fc-b8ea-492f-b861-2ff1a09095d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:08:35.958889 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:35.958877    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/008158fc-b8ea-492f-b861-2ff1a09095d7-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:08:36.770635 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:36.770600    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"
Sep 18 19:08:36.770807 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:36.770648    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6" event={"ID":"008158fc-b8ea-492f-b861-2ff1a09095d7","Type":"ContainerDied","Data":"6a401d8cff7db9382b0bd16b395e665453d82ce2e5f483e9a4842961f0717846"}
Sep 18 19:08:36.770807 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:36.770694    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a401d8cff7db9382b0bd16b395e665453d82ce2e5f483e9a4842961f0717846"
Sep 18 19:08:49.393656 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.393617    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"]
Sep 18 19:08:49.394259 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.393963    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t" podUID="2918dfe3-682e-4f11-887d-e31309836e9f" containerName="payload-processing" containerID="cri-o://4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8" gracePeriod=30
Sep 18 19:08:49.676508 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.676479    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:08:49.798238 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.798199    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2918dfe3-682e-4f11-887d-e31309836e9f-plugins-config-volume\") pod \"2918dfe3-682e-4f11-887d-e31309836e9f\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") "
Sep 18 19:08:49.798404 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.798311    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx6fs\" (UniqueName: \"kubernetes.io/projected/2918dfe3-682e-4f11-887d-e31309836e9f-kube-api-access-tx6fs\") pod \"2918dfe3-682e-4f11-887d-e31309836e9f\" (UID: \"2918dfe3-682e-4f11-887d-e31309836e9f\") "
Sep 18 19:08:49.798621 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.798593    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2918dfe3-682e-4f11-887d-e31309836e9f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2918dfe3-682e-4f11-887d-e31309836e9f" (UID: "2918dfe3-682e-4f11-887d-e31309836e9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:08:49.798714 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.798702    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2918dfe3-682e-4f11-887d-e31309836e9f-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:08:49.800362 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.800341    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2918dfe3-682e-4f11-887d-e31309836e9f-kube-api-access-tx6fs" (OuterVolumeSpecName: "kube-api-access-tx6fs") pod "2918dfe3-682e-4f11-887d-e31309836e9f" (UID: "2918dfe3-682e-4f11-887d-e31309836e9f"). InnerVolumeSpecName "kube-api-access-tx6fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:08:49.846797 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.846760    2568 generic.go:356] "Generic (PLEG): container finished" podID="2918dfe3-682e-4f11-887d-e31309836e9f" containerID="4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8" exitCode=0
Sep 18 19:08:49.846969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.846802    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"
Sep 18 19:08:49.846969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.846815    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t" event={"ID":"2918dfe3-682e-4f11-887d-e31309836e9f","Type":"ContainerDied","Data":"4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8"}
Sep 18 19:08:49.846969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.846893    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t" event={"ID":"2918dfe3-682e-4f11-887d-e31309836e9f","Type":"ContainerDied","Data":"f9079feca336600c3b629754f42a3aa1a010b1712d12c60e9819717c6a6e128e"}
Sep 18 19:08:49.846969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.846925    2568 scope.go:117] "RemoveContainer" containerID="4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8"
Sep 18 19:08:49.857312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.857290    2568 scope.go:117] "RemoveContainer" containerID="4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8"
Sep 18 19:08:49.857619 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:08:49.857599    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8\": container with ID starting with 4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8 not found: ID does not exist" containerID="4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8"
Sep 18 19:08:49.857691 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.857633    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8"} err="failed to get container status \"4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8\": rpc error: code = NotFound desc = could not find container \"4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8\": container with ID starting with 4b3c51a3f349e62b4894c243c12587606bd30898dc8e7cf79f7ec5bc6f32e1e8 not found: ID does not exist"
Sep 18 19:08:49.894471 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.894443    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"]
Sep 18 19:08:49.898164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.898141    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4ac9ac7d-56bcd4ddf-b8v6t"]
Sep 18 19:08:49.899339 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.899321    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx6fs\" (UniqueName: \"kubernetes.io/projected/2918dfe3-682e-4f11-887d-e31309836e9f-kube-api-access-tx6fs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:08:49.982603 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.982571    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"]
Sep 18 19:08:49.991527 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:49.991481    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ac9ac7d-6dwj6"]
Sep 18 19:08:51.055741 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:51.055708    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="008158fc-b8ea-492f-b861-2ff1a09095d7" path="/var/lib/kubelet/pods/008158fc-b8ea-492f-b861-2ff1a09095d7/volumes"
Sep 18 19:08:51.056105 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:08:51.056072    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2918dfe3-682e-4f11-887d-e31309836e9f" path="/var/lib/kubelet/pods/2918dfe3-682e-4f11-887d-e31309836e9f/volumes"
Sep 18 19:09:31.082110 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:09:31.082079    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:09:31.087640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:09:31.087614    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:10:31.285635 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:10:31.285601    2568 scope.go:117] "RemoveContainer" containerID="979fab9a7e9e40c02cc23e752e364db8f3d3ea7b2b1476ae15da4953efe9dfe6"
Sep 18 19:10:31.294494 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:10:31.294472    2568 scope.go:117] "RemoveContainer" containerID="4716f5092f65bba6329c25e79ca61844114ab7e3285461bae07e17befea9d49c"
Sep 18 19:10:31.302379 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:10:31.302360    2568 scope.go:117] "RemoveContainer" containerID="080840821ec9404de7d61bc6652cf6be3c8b77dea14fda7ec18703774571aef8"
Sep 18 19:11:00.505805 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.505769    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"]
Sep 18 19:11:00.506255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.506224    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2918dfe3-682e-4f11-887d-e31309836e9f" containerName="payload-processing"
Sep 18 19:11:00.506255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.506235    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2918dfe3-682e-4f11-887d-e31309836e9f" containerName="payload-processing"
Sep 18 19:11:00.506255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.506246    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="008158fc-b8ea-492f-b861-2ff1a09095d7" containerName="revoke-keys"
Sep 18 19:11:00.506255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.506251    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="008158fc-b8ea-492f-b861-2ff1a09095d7" containerName="revoke-keys"
Sep 18 19:11:00.506389 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.506329    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2918dfe3-682e-4f11-887d-e31309836e9f" containerName="payload-processing"
Sep 18 19:11:00.506389 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.506336    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="008158fc-b8ea-492f-b861-2ff1a09095d7" containerName="revoke-keys"
Sep 18 19:11:00.509555 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.509536    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.526755 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.526734    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1d6196-openshift-default-dockercfg-2f982\""
Sep 18 19:11:00.536204 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536179    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-certs\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536226    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs68h\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-kube-api-access-bs68h\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536258    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-podinfo\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536301    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-envoy\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-socket\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536337    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-token\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536354    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-credential-socket\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536378    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2004ed50-4110-4eb9-b8e6-8516a188544f-istiod-ca-cert\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.536576 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.536453    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-data\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.606386 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.606352    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"]
Sep 18 19:11:00.637210 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs68h\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-kube-api-access-bs68h\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637386 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637238    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-podinfo\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637386 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-envoy\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637386 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637319    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-socket\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637386 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-token\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637386 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637373    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-credential-socket\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637682 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637417    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2004ed50-4110-4eb9-b8e6-8516a188544f-istiod-ca-cert\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637682 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637444    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-data\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637682 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637581    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-certs\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637847 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-socket\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637904 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637866    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-credential-socket\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.637980 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637961    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-certs\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.638042 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.637963    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-data\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.638245 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.638225    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2004ed50-4110-4eb9-b8e6-8516a188544f-istiod-ca-cert\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.639691 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.639666    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-envoy\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.639781 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.639749    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-podinfo\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.670275 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.670247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs68h\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-kube-api-access-bs68h\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.672761 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.672743    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-token\") pod \"e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") " pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:00.819558 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:00.819442    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:01.139300 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.139272    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"]
Sep 18 19:11:01.141038 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:11:01.141014    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2004ed50_4110_4eb9_b8e6_8516a188544f.slice/crio-f6b21603c9c534156aab8fb8f7b65555215105649645d3f53be0f4a255fcb2b4 WatchSource:0}: Error finding container f6b21603c9c534156aab8fb8f7b65555215105649645d3f53be0f4a255fcb2b4: Status 404 returned error can't find the container with id f6b21603c9c534156aab8fb8f7b65555215105649645d3f53be0f4a255fcb2b4
Sep 18 19:11:01.143489 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.143443    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:11:01.143577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.143555    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:11:01.143618 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.143608    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:11:01.606153 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.606119    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" event={"ID":"2004ed50-4110-4eb9-b8e6-8516a188544f","Type":"ContainerStarted","Data":"1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"}
Sep 18 19:11:01.606562 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.606175    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" event={"ID":"2004ed50-4110-4eb9-b8e6-8516a188544f","Type":"ContainerStarted","Data":"f6b21603c9c534156aab8fb8f7b65555215105649645d3f53be0f4a255fcb2b4"}
Sep 18 19:11:01.663121 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.663053    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podStartSLOduration=1.6630323740000001 podStartE2EDuration="1.663032374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:11:01.660050551 +0000 UTC m=+1591.098310780" watchObservedRunningTime="2026-09-18 19:11:01.663032374 +0000 UTC m=+1591.101292579"
Sep 18 19:11:01.820263 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:01.820229    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:02.825456 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:02.825424    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:03.616484 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:03.616451    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:03.617560 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:03.617542    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:15.441817 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.441777    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"]
Sep 18 19:11:15.445494 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.445479    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.448154 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.448132    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1d6196-dockercfg-mhvjx\""
Sep 18 19:11:15.448353 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.448338    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1d6196\""
Sep 18 19:11:15.462500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.462477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"]
Sep 18 19:11:15.593785 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.593754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-459ph\" (UniqueName: \"kubernetes.io/projected/72f4bb93-2dae-4719-ba02-d47a786114b3-kube-api-access-459ph\") pod \"payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.593957 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.593868    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72f4bb93-2dae-4719-ba02-d47a786114b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.694891 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.694807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72f4bb93-2dae-4719-ba02-d47a786114b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.695039 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.694914    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-459ph\" (UniqueName: \"kubernetes.io/projected/72f4bb93-2dae-4719-ba02-d47a786114b3-kube-api-access-459ph\") pod \"payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.695565 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.695544    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72f4bb93-2dae-4719-ba02-d47a786114b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.706474 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.706447    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-459ph\" (UniqueName: \"kubernetes.io/projected/72f4bb93-2dae-4719-ba02-d47a786114b3-kube-api-access-459ph\") pod \"payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:15.756686 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:15.756657    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:16.051894 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:16.051847    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"]
Sep 18 19:11:16.054323 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:11:16.054297    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f4bb93_2dae_4719_ba02_d47a786114b3.slice/crio-25653a11afe167280ec568b6cb1d14496fee567d86cbd60a82e7deede924fb8a WatchSource:0}: Error finding container 25653a11afe167280ec568b6cb1d14496fee567d86cbd60a82e7deede924fb8a: Status 404 returned error can't find the container with id 25653a11afe167280ec568b6cb1d14496fee567d86cbd60a82e7deede924fb8a
Sep 18 19:11:16.690065 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:16.690030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf" event={"ID":"72f4bb93-2dae-4719-ba02-d47a786114b3","Type":"ContainerStarted","Data":"f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507"}
Sep 18 19:11:16.690477 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:16.690092    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:16.690477 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:16.690108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf" event={"ID":"72f4bb93-2dae-4719-ba02-d47a786114b3","Type":"ContainerStarted","Data":"25653a11afe167280ec568b6cb1d14496fee567d86cbd60a82e7deede924fb8a"}
Sep 18 19:11:16.714481 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:16.714426    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf" podStartSLOduration=1.714409291 podStartE2EDuration="1.714409291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:11:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:11:16.712411944 +0000 UTC m=+1606.150672148" watchObservedRunningTime="2026-09-18 19:11:16.714409291 +0000 UTC m=+1606.152669494"
Sep 18 19:11:27.698098 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:27.698064    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:11:37.021680 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.021643    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"]
Sep 18 19:11:37.024443 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.024426    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:37.028072 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.028051    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:11:37.037664 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.037643    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:11:37.056420 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.056390    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"]
Sep 18 19:11:37.117135 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.117103    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e0a2057-4dac-4669-9aad-8dd43a61fa5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf\" (UID: \"2e0a2057-4dac-4669-9aad-8dd43a61fa5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:37.218343 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.218304    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e0a2057-4dac-4669-9aad-8dd43a61fa5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf\" (UID: \"2e0a2057-4dac-4669-9aad-8dd43a61fa5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:37.219001 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.218973    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e0a2057-4dac-4669-9aad-8dd43a61fa5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf\" (UID: \"2e0a2057-4dac-4669-9aad-8dd43a61fa5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:37.335684 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.335591    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:37.643436 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.643405    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"]
Sep 18 19:11:37.645106 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:11:37.645078    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0a2057_4dac_4669_9aad_8dd43a61fa5c.slice/crio-174d829d3be7d30d03d22f5eb2690a2e1f6af3bf7468db79cc06ecb4828a155b WatchSource:0}: Error finding container 174d829d3be7d30d03d22f5eb2690a2e1f6af3bf7468db79cc06ecb4828a155b: Status 404 returned error can't find the container with id 174d829d3be7d30d03d22f5eb2690a2e1f6af3bf7468db79cc06ecb4828a155b
Sep 18 19:11:37.821845 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:37.821809    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf" event={"ID":"2e0a2057-4dac-4669-9aad-8dd43a61fa5c","Type":"ContainerStarted","Data":"174d829d3be7d30d03d22f5eb2690a2e1f6af3bf7468db79cc06ecb4828a155b"}
Sep 18 19:11:38.829974 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:38.829933    2568 generic.go:356] "Generic (PLEG): container finished" podID="2e0a2057-4dac-4669-9aad-8dd43a61fa5c" containerID="46d4874745a34f51493e389dc79018b41948b4a1e8e835e1a62e30d5a8f9c00f" exitCode=0
Sep 18 19:11:38.830408 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:38.830021    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf" event={"ID":"2e0a2057-4dac-4669-9aad-8dd43a61fa5c","Type":"ContainerDied","Data":"46d4874745a34f51493e389dc79018b41948b4a1e8e835e1a62e30d5a8f9c00f"}
Sep 18 19:11:39.838040 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:39.838005    2568 generic.go:356] "Generic (PLEG): container finished" podID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerID="1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6" exitCode=0
Sep 18 19:11:39.838453 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:39.838074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" event={"ID":"2004ed50-4110-4eb9-b8e6-8516a188544f","Type":"ContainerDied","Data":"1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"}
Sep 18 19:11:39.838917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:39.838627    2568 scope.go:117] "RemoveContainer" containerID="1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"
Sep 18 19:11:39.839813 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:39.839781    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:11:39.839943 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:39.839847    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:11:39.839943 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:39.839878    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:11:40.012080 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.012010    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:40.053191 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.053161    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e0a2057-4dac-4669-9aad-8dd43a61fa5c-maas-api-service-ca\") pod \"2e0a2057-4dac-4669-9aad-8dd43a61fa5c\" (UID: \"2e0a2057-4dac-4669-9aad-8dd43a61fa5c\") "
Sep 18 19:11:40.053542 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.053490    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e0a2057-4dac-4669-9aad-8dd43a61fa5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e0a2057-4dac-4669-9aad-8dd43a61fa5c" (UID: "2e0a2057-4dac-4669-9aad-8dd43a61fa5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:11:40.154301 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.154271    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e0a2057-4dac-4669-9aad-8dd43a61fa5c-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:11:40.846809 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.846776    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" event={"ID":"2004ed50-4110-4eb9-b8e6-8516a188544f","Type":"ContainerStarted","Data":"adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3"}
Sep 18 19:11:40.848369 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.848348    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf" event={"ID":"2e0a2057-4dac-4669-9aad-8dd43a61fa5c","Type":"ContainerDied","Data":"174d829d3be7d30d03d22f5eb2690a2e1f6af3bf7468db79cc06ecb4828a155b"}
Sep 18 19:11:40.848488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.848365    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"
Sep 18 19:11:40.848488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:40.848388    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="174d829d3be7d30d03d22f5eb2690a2e1f6af3bf7468db79cc06ecb4828a155b"
Sep 18 19:11:41.820460 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:41.820426    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:41.820677 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:41.820654    2568 patch_prober.go:28] interesting pod/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 18 19:11:41.820728 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:41.820706    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 18 19:11:42.820342 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:42.820303    2568 patch_prober.go:28] interesting pod/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 18 19:11:42.820754 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:42.820367    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 18 19:11:43.820490 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:43.820454    2568 patch_prober.go:28] interesting pod/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 18 19:11:43.820990 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:43.820551    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 18 19:11:44.820682 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:44.820648    2568 patch_prober.go:28] interesting pod/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 18 19:11:44.821088 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:44.820700    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 18 19:11:45.820531 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:45.820470    2568 patch_prober.go:28] interesting pod/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 18 19:11:45.820775 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:45.820555    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 18 19:11:46.823905 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:46.823871    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:46.824363 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:46.824169    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:46.825023 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:46.825007    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:11:53.399445 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.399405    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"]
Sep 18 19:11:53.399995 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.399980    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0a2057-4dac-4669-9aad-8dd43a61fa5c" containerName="revoke-keys"
Sep 18 19:11:53.399995 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.399996    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0a2057-4dac-4669-9aad-8dd43a61fa5c" containerName="revoke-keys"
Sep 18 19:11:53.400091 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.400081    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e0a2057-4dac-4669-9aad-8dd43a61fa5c" containerName="revoke-keys"
Sep 18 19:11:53.402859 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.402842    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.408103 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.408076    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-60d8bd\""
Sep 18 19:11:53.408238 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.408179    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-60d8bd-dockercfg-h4gdh\""
Sep 18 19:11:53.432663 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.432634    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"]
Sep 18 19:11:53.512862 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.512829    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v99hg\" (UniqueName: \"kubernetes.io/projected/d043091e-4018-4a30-85b9-bc1f880a2a75-kube-api-access-v99hg\") pod \"payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.513048 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.512902    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d043091e-4018-4a30-85b9-bc1f880a2a75-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.529074 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.529036    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"]
Sep 18 19:11:53.532222 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.532199    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.554217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.554188    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"]
Sep 18 19:11:53.613818 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.613784    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn285\" (UniqueName: \"kubernetes.io/projected/aff1d110-ac96-4a89-bc91-c99527600b1f-kube-api-access-zn285\") pod \"payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") " pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.614016 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.613838    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff1d110-ac96-4a89-bc91-c99527600b1f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") " pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.614016 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.613976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v99hg\" (UniqueName: \"kubernetes.io/projected/d043091e-4018-4a30-85b9-bc1f880a2a75-kube-api-access-v99hg\") pod \"payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.614142 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.614073    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d043091e-4018-4a30-85b9-bc1f880a2a75-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.614818 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.614799    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d043091e-4018-4a30-85b9-bc1f880a2a75-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.634111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.634081    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v99hg\" (UniqueName: \"kubernetes.io/projected/d043091e-4018-4a30-85b9-bc1f880a2a75-kube-api-access-v99hg\") pod \"payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.713164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.713076    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:53.715823 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.715505    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn285\" (UniqueName: \"kubernetes.io/projected/aff1d110-ac96-4a89-bc91-c99527600b1f-kube-api-access-zn285\") pod \"payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") " pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.715823 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.715595    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff1d110-ac96-4a89-bc91-c99527600b1f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") " pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.716262 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.716243    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff1d110-ac96-4a89-bc91-c99527600b1f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") " pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.737259 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.737228    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn285\" (UniqueName: \"kubernetes.io/projected/aff1d110-ac96-4a89-bc91-c99527600b1f-kube-api-access-zn285\") pod \"payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") " pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:53.842309 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:53.842272    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:54.001553 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.001500    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"]
Sep 18 19:11:54.003128 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:11:54.003104    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd043091e_4018_4a30_85b9_bc1f880a2a75.slice/crio-93029a8bdc9ecf625a8d7aa38a4538a34e107da2506c584581dbaa0309ede41e WatchSource:0}: Error finding container 93029a8bdc9ecf625a8d7aa38a4538a34e107da2506c584581dbaa0309ede41e: Status 404 returned error can't find the container with id 93029a8bdc9ecf625a8d7aa38a4538a34e107da2506c584581dbaa0309ede41e
Sep 18 19:11:54.122237 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.121304    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"]
Sep 18 19:11:54.127289 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:11:54.127256    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff1d110_ac96_4a89_bc91_c99527600b1f.slice/crio-f3316c94454f9b3c1a06186e80e677a53d401c42d642a66bff30321084a494f7 WatchSource:0}: Error finding container f3316c94454f9b3c1a06186e80e677a53d401c42d642a66bff30321084a494f7: Status 404 returned error can't find the container with id f3316c94454f9b3c1a06186e80e677a53d401c42d642a66bff30321084a494f7
Sep 18 19:11:54.934167 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.934123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz" event={"ID":"aff1d110-ac96-4a89-bc91-c99527600b1f","Type":"ContainerStarted","Data":"3a86211ed12ff0d25cbb9a32059f48031a1abb19bd3cbed8261615474245d489"}
Sep 18 19:11:54.934676 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.934333    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:11:54.934676 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.934383    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz" event={"ID":"aff1d110-ac96-4a89-bc91-c99527600b1f","Type":"ContainerStarted","Data":"f3316c94454f9b3c1a06186e80e677a53d401c42d642a66bff30321084a494f7"}
Sep 18 19:11:54.935897 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.935872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2" event={"ID":"d043091e-4018-4a30-85b9-bc1f880a2a75","Type":"ContainerStarted","Data":"5a2daebe7a85a563e3dfe2afbe49238efd2718315ef66ee60b3baaa9847f8937"}
Sep 18 19:11:54.935997 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.935916    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2" event={"ID":"d043091e-4018-4a30-85b9-bc1f880a2a75","Type":"ContainerStarted","Data":"93029a8bdc9ecf625a8d7aa38a4538a34e107da2506c584581dbaa0309ede41e"}
Sep 18 19:11:54.935997 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.935952    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:11:54.969904 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.969845    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz" podStartSLOduration=1.969827167 podStartE2EDuration="1.969827167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:11:54.966255724 +0000 UTC m=+1644.404515928" watchObservedRunningTime="2026-09-18 19:11:54.969827167 +0000 UTC m=+1644.408087375"
Sep 18 19:11:54.993307 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:54.993247    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2" podStartSLOduration=1.993226645 podStartE2EDuration="1.993226645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:11:54.990704539 +0000 UTC m=+1644.428964746" watchObservedRunningTime="2026-09-18 19:11:54.993226645 +0000 UTC m=+1644.431486851"
Sep 18 19:11:58.216718 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.216683    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"]
Sep 18 19:11:58.217171 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.216927    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5" podUID="c7b84adb-d4de-498f-bcc5-13ad6ac6939f" containerName="payload-pre-processing" containerID="cri-o://406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7" gracePeriod=30
Sep 18 19:11:58.483556 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.483530    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"]
Sep 18 19:11:58.488506 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.488457    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1a444c-ghlnf"]
Sep 18 19:11:58.506650 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.506625    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 19:11:58.665767 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.665731    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bj8m\" (UniqueName: \"kubernetes.io/projected/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-kube-api-access-7bj8m\") pod \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") "
Sep 18 19:11:58.665962 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.665801    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-plugins-config-volume\") pod \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\" (UID: \"c7b84adb-d4de-498f-bcc5-13ad6ac6939f\") "
Sep 18 19:11:58.666128 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.666105    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7b84adb-d4de-498f-bcc5-13ad6ac6939f" (UID: "c7b84adb-d4de-498f-bcc5-13ad6ac6939f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:11:58.667882 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.667854    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-kube-api-access-7bj8m" (OuterVolumeSpecName: "kube-api-access-7bj8m") pod "c7b84adb-d4de-498f-bcc5-13ad6ac6939f" (UID: "c7b84adb-d4de-498f-bcc5-13ad6ac6939f"). InnerVolumeSpecName "kube-api-access-7bj8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:11:58.767574 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.767451    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bj8m\" (UniqueName: \"kubernetes.io/projected/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-kube-api-access-7bj8m\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:11:58.767574 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.767485    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b84adb-d4de-498f-bcc5-13ad6ac6939f-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:11:58.962246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.962209    2568 generic.go:356] "Generic (PLEG): container finished" podID="c7b84adb-d4de-498f-bcc5-13ad6ac6939f" containerID="406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7" exitCode=0
Sep 18 19:11:58.962396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.962255    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"
Sep 18 19:11:58.962396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.962299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5" event={"ID":"c7b84adb-d4de-498f-bcc5-13ad6ac6939f","Type":"ContainerDied","Data":"406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7"}
Sep 18 19:11:58.962396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.962357    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5" event={"ID":"c7b84adb-d4de-498f-bcc5-13ad6ac6939f","Type":"ContainerDied","Data":"474e149678dc451c16a62e1959ddff8e8e39cc300cf9e3b53ca5fc6594adf3ba"}
Sep 18 19:11:58.962396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.962388    2568 scope.go:117] "RemoveContainer" containerID="406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7"
Sep 18 19:11:58.972425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.972406    2568 scope.go:117] "RemoveContainer" containerID="406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7"
Sep 18 19:11:58.972709 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:11:58.972692    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7\": container with ID starting with 406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7 not found: ID does not exist" containerID="406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7"
Sep 18 19:11:58.972766 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.972717    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7"} err="failed to get container status \"406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7\": rpc error: code = NotFound desc = could not find container \"406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7\": container with ID starting with 406995f6e73e381dcc8629fc35142ca0dba59f67fc85141a2f9aeb25f5bf84c7 not found: ID does not exist"
Sep 18 19:11:58.989099 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:58.989075    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"]
Sep 18 19:11:59.002835 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:59.002810    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1a444c-9d7458574-6xtj5"]
Sep 18 19:11:59.055244 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:59.055169    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e0a2057-4dac-4669-9aad-8dd43a61fa5c" path="/var/lib/kubelet/pods/2e0a2057-4dac-4669-9aad-8dd43a61fa5c/volumes"
Sep 18 19:11:59.055592 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:11:59.055578    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7b84adb-d4de-498f-bcc5-13ad6ac6939f" path="/var/lib/kubelet/pods/c7b84adb-d4de-498f-bcc5-13ad6ac6939f/volumes"
Sep 18 19:12:05.944009 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:05.943978    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:12:05.944423 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:05.944212    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:12:11.949084 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.949043    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"]
Sep 18 19:12:11.949589 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.949574    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b84adb-d4de-498f-bcc5-13ad6ac6939f" containerName="payload-pre-processing"
Sep 18 19:12:11.949643 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.949591    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b84adb-d4de-498f-bcc5-13ad6ac6939f" containerName="payload-pre-processing"
Sep 18 19:12:11.949679 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.949673    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b84adb-d4de-498f-bcc5-13ad6ac6939f" containerName="payload-pre-processing"
Sep 18 19:12:11.953202 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.953184    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:11.956552 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.956509    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:12:11.956802 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.956783    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:12:11.969877 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:11.969843    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"]
Sep 18 19:12:12.004919 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:12.004875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a71e-1a72-410c-afa0-a60a78eafd19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2\" (UID: \"62c1a71e-1a72-410c-afa0-a60a78eafd19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:12.105905 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:12.105871    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a71e-1a72-410c-afa0-a60a78eafd19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2\" (UID: \"62c1a71e-1a72-410c-afa0-a60a78eafd19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:12.106673 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:12.106655    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a71e-1a72-410c-afa0-a60a78eafd19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2\" (UID: \"62c1a71e-1a72-410c-afa0-a60a78eafd19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:12.263698 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:12.263663    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:12.556561 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:12.556535    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"]
Sep 18 19:12:12.558542 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:12:12.558503    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c1a71e_1a72_410c_afa0_a60a78eafd19.slice/crio-83c83758bebee4207489e4ee13a2f3486a91f7f863dc4cc636409f72bedf7cfd WatchSource:0}: Error finding container 83c83758bebee4207489e4ee13a2f3486a91f7f863dc4cc636409f72bedf7cfd: Status 404 returned error can't find the container with id 83c83758bebee4207489e4ee13a2f3486a91f7f863dc4cc636409f72bedf7cfd
Sep 18 19:12:13.054983 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:13.054951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2" event={"ID":"62c1a71e-1a72-410c-afa0-a60a78eafd19","Type":"ContainerStarted","Data":"c11ac4449f85b76bfcbfc26bd2a09170414615b164845a8e7774d2dd9c9c9043"}
Sep 18 19:12:13.055366 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:13.054997    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2" event={"ID":"62c1a71e-1a72-410c-afa0-a60a78eafd19","Type":"ContainerStarted","Data":"83c83758bebee4207489e4ee13a2f3486a91f7f863dc4cc636409f72bedf7cfd"}
Sep 18 19:12:13.075270 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:13.075208    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2" podStartSLOduration=2.075193576 podStartE2EDuration="2.075193576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:12:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:12:13.073852777 +0000 UTC m=+1662.512112981" watchObservedRunningTime="2026-09-18 19:12:13.075193576 +0000 UTC m=+1662.513453780"
Sep 18 19:12:14.061092 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:14.061062    2568 generic.go:356] "Generic (PLEG): container finished" podID="62c1a71e-1a72-410c-afa0-a60a78eafd19" containerID="c11ac4449f85b76bfcbfc26bd2a09170414615b164845a8e7774d2dd9c9c9043" exitCode=0
Sep 18 19:12:14.061464 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:14.061146    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2" event={"ID":"62c1a71e-1a72-410c-afa0-a60a78eafd19","Type":"ContainerDied","Data":"c11ac4449f85b76bfcbfc26bd2a09170414615b164845a8e7774d2dd9c9c9043"}
Sep 18 19:12:15.217746 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:15.217720    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:15.341029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:15.340931    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a71e-1a72-410c-afa0-a60a78eafd19-maas-api-service-ca\") pod \"62c1a71e-1a72-410c-afa0-a60a78eafd19\" (UID: \"62c1a71e-1a72-410c-afa0-a60a78eafd19\") "
Sep 18 19:12:15.341342 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:15.341318    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c1a71e-1a72-410c-afa0-a60a78eafd19-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62c1a71e-1a72-410c-afa0-a60a78eafd19" (UID: "62c1a71e-1a72-410c-afa0-a60a78eafd19"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:15.441982 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:15.441934    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a71e-1a72-410c-afa0-a60a78eafd19-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:16.073227 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:16.073199    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"
Sep 18 19:12:16.073417 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:16.073226    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2" event={"ID":"62c1a71e-1a72-410c-afa0-a60a78eafd19","Type":"ContainerDied","Data":"83c83758bebee4207489e4ee13a2f3486a91f7f863dc4cc636409f72bedf7cfd"}
Sep 18 19:12:16.073417 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:16.073280    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83c83758bebee4207489e4ee13a2f3486a91f7f863dc4cc636409f72bedf7cfd"
Sep 18 19:12:28.076304 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.076262    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"]
Sep 18 19:12:28.076835 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.076551    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf" podUID="72f4bb93-2dae-4719-ba02-d47a786114b3" containerName="payload-pre-processing" containerID="cri-o://f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507" gracePeriod=30
Sep 18 19:12:28.283723 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.283687    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"]
Sep 18 19:12:28.288556 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.288509    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6196-h5ds2"]
Sep 18 19:12:28.350295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.350268    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:12:28.490665 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.490625    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-459ph\" (UniqueName: \"kubernetes.io/projected/72f4bb93-2dae-4719-ba02-d47a786114b3-kube-api-access-459ph\") pod \"72f4bb93-2dae-4719-ba02-d47a786114b3\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") "
Sep 18 19:12:28.490665 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.490675    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72f4bb93-2dae-4719-ba02-d47a786114b3-plugins-config-volume\") pod \"72f4bb93-2dae-4719-ba02-d47a786114b3\" (UID: \"72f4bb93-2dae-4719-ba02-d47a786114b3\") "
Sep 18 19:12:28.491093 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.491065    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f4bb93-2dae-4719-ba02-d47a786114b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72f4bb93-2dae-4719-ba02-d47a786114b3" (UID: "72f4bb93-2dae-4719-ba02-d47a786114b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:28.492861 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.492831    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f4bb93-2dae-4719-ba02-d47a786114b3-kube-api-access-459ph" (OuterVolumeSpecName: "kube-api-access-459ph") pod "72f4bb93-2dae-4719-ba02-d47a786114b3" (UID: "72f4bb93-2dae-4719-ba02-d47a786114b3"). InnerVolumeSpecName "kube-api-access-459ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:28.591938 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.591837    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-459ph\" (UniqueName: \"kubernetes.io/projected/72f4bb93-2dae-4719-ba02-d47a786114b3-kube-api-access-459ph\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:28.591938 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:28.591876    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72f4bb93-2dae-4719-ba02-d47a786114b3-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:29.055532 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.055481    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c1a71e-1a72-410c-afa0-a60a78eafd19" path="/var/lib/kubelet/pods/62c1a71e-1a72-410c-afa0-a60a78eafd19/volumes"
Sep 18 19:12:29.155785 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.155747    2568 generic.go:356] "Generic (PLEG): container finished" podID="72f4bb93-2dae-4719-ba02-d47a786114b3" containerID="f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507" exitCode=0
Sep 18 19:12:29.156196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.155816    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"
Sep 18 19:12:29.156196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.155843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf" event={"ID":"72f4bb93-2dae-4719-ba02-d47a786114b3","Type":"ContainerDied","Data":"f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507"}
Sep 18 19:12:29.156196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.155900    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf" event={"ID":"72f4bb93-2dae-4719-ba02-d47a786114b3","Type":"ContainerDied","Data":"25653a11afe167280ec568b6cb1d14496fee567d86cbd60a82e7deede924fb8a"}
Sep 18 19:12:29.156196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.155927    2568 scope.go:117] "RemoveContainer" containerID="f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507"
Sep 18 19:12:29.165340 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.165319    2568 scope.go:117] "RemoveContainer" containerID="f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507"
Sep 18 19:12:29.165692 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:12:29.165669    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507\": container with ID starting with f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507 not found: ID does not exist" containerID="f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507"
Sep 18 19:12:29.165746 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.165705    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507"} err="failed to get container status \"f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507\": rpc error: code = NotFound desc = could not find container \"f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507\": container with ID starting with f96fe45c3ef59347afb8342538e08ae8ab43a9b326b528dc98ba721f07c57507 not found: ID does not exist"
Sep 18 19:12:29.178600 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.178571    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"]
Sep 18 19:12:29.183980 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:29.183953    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6196-5c7b57bd9d-d9wxf"]
Sep 18 19:12:31.055660 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:31.055632    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f4bb93-2dae-4719-ba02-d47a786114b3" path="/var/lib/kubelet/pods/72f4bb93-2dae-4719-ba02-d47a786114b3/volumes"
Sep 18 19:12:31.343166 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:31.343086    2568 scope.go:117] "RemoveContainer" containerID="5a5c5a073812ee7d9cd144b9b9007b908d3eebbea45adbf3898820b9a24360e5"
Sep 18 19:12:38.958753 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:38.958714    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"]
Sep 18 19:12:38.959287 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:38.958953    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy" containerID="cri-o://adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3" gracePeriod=30
Sep 18 19:12:39.336046 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336013    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"]
Sep 18 19:12:39.336498 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336486    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c1a71e-1a72-410c-afa0-a60a78eafd19" containerName="revoke-keys"
Sep 18 19:12:39.336573 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336499    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c1a71e-1a72-410c-afa0-a60a78eafd19" containerName="revoke-keys"
Sep 18 19:12:39.336573 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336536    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f4bb93-2dae-4719-ba02-d47a786114b3" containerName="payload-pre-processing"
Sep 18 19:12:39.336573 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336546    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f4bb93-2dae-4719-ba02-d47a786114b3" containerName="payload-pre-processing"
Sep 18 19:12:39.336685 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336628    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c1a71e-1a72-410c-afa0-a60a78eafd19" containerName="revoke-keys"
Sep 18 19:12:39.336685 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.336645    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f4bb93-2dae-4719-ba02-d47a786114b3" containerName="payload-pre-processing"
Sep 18 19:12:39.340064 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.340043    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:39.344997 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.344981    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:12:39.345088 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.344981    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:12:39.351839 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.351818    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"]
Sep 18 19:12:39.405686 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.405654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9290a3-81cf-417f-9486-14ab965cdb75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn\" (UID: \"0b9290a3-81cf-417f-9486-14ab965cdb75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:39.507061 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.507023    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9290a3-81cf-417f-9486-14ab965cdb75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn\" (UID: \"0b9290a3-81cf-417f-9486-14ab965cdb75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:39.507674 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.507654    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9290a3-81cf-417f-9486-14ab965cdb75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn\" (UID: \"0b9290a3-81cf-417f-9486-14ab965cdb75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:39.651804 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:39.651717    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:40.014495 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:40.014471    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"]
Sep 18 19:12:40.015592 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:12:40.015568    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9290a3_81cf_417f_9486_14ab965cdb75.slice/crio-39969cde9c7c05ad905b0d0ea27dc950b099e1c0779ed1b3ffe461d165b49006 WatchSource:0}: Error finding container 39969cde9c7c05ad905b0d0ea27dc950b099e1c0779ed1b3ffe461d165b49006: Status 404 returned error can't find the container with id 39969cde9c7c05ad905b0d0ea27dc950b099e1c0779ed1b3ffe461d165b49006
Sep 18 19:12:40.225212 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:40.225178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn" event={"ID":"0b9290a3-81cf-417f-9486-14ab965cdb75","Type":"ContainerStarted","Data":"39969cde9c7c05ad905b0d0ea27dc950b099e1c0779ed1b3ffe461d165b49006"}
Sep 18 19:12:41.236287 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:41.236254    2568 generic.go:356] "Generic (PLEG): container finished" podID="0b9290a3-81cf-417f-9486-14ab965cdb75" containerID="8348b0324c5df765998a03679b13fa2eb15c8a4c240ca47559452d2981e1280e" exitCode=0
Sep 18 19:12:41.236773 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:41.236380    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn" event={"ID":"0b9290a3-81cf-417f-9486-14ab965cdb75","Type":"ContainerDied","Data":"8348b0324c5df765998a03679b13fa2eb15c8a4c240ca47559452d2981e1280e"}
Sep 18 19:12:42.389110 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:42.389083    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:42.439701 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:42.439671    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9290a3-81cf-417f-9486-14ab965cdb75-maas-api-service-ca\") pod \"0b9290a3-81cf-417f-9486-14ab965cdb75\" (UID: \"0b9290a3-81cf-417f-9486-14ab965cdb75\") "
Sep 18 19:12:42.440023 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:42.440001    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9290a3-81cf-417f-9486-14ab965cdb75-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b9290a3-81cf-417f-9486-14ab965cdb75" (UID: "0b9290a3-81cf-417f-9486-14ab965cdb75"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:42.541610 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:42.541501    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9290a3-81cf-417f-9486-14ab965cdb75-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:43.249319 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:43.249292    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"
Sep 18 19:12:43.249319 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:43.249304    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn" event={"ID":"0b9290a3-81cf-417f-9486-14ab965cdb75","Type":"ContainerDied","Data":"39969cde9c7c05ad905b0d0ea27dc950b099e1c0779ed1b3ffe461d165b49006"}
Sep 18 19:12:43.249550 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:43.249350    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39969cde9c7c05ad905b0d0ea27dc950b099e1c0779ed1b3ffe461d165b49006"
Sep 18 19:12:44.216992 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.216964    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:12:44.256747 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.256667    2568 generic.go:356] "Generic (PLEG): container finished" podID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerID="adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3" exitCode=0
Sep 18 19:12:44.256747 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.256704    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" event={"ID":"2004ed50-4110-4eb9-b8e6-8516a188544f","Type":"ContainerDied","Data":"adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3"}
Sep 18 19:12:44.256747 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.256729    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"
Sep 18 19:12:44.257091 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.256768    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w" event={"ID":"2004ed50-4110-4eb9-b8e6-8516a188544f","Type":"ContainerDied","Data":"f6b21603c9c534156aab8fb8f7b65555215105649645d3f53be0f4a255fcb2b4"}
Sep 18 19:12:44.257091 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.256792    2568 scope.go:117] "RemoveContainer" containerID="adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3"
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258715    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-data\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258755    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-certs\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258810    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs68h\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-kube-api-access-bs68h\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258857    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-token\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258886    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-socket\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258972    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2004ed50-4110-4eb9-b8e6-8516a188544f-istiod-ca-cert\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.258995    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-envoy\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.259016    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-credential-socket\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.259566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.259106    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-podinfo\") pod \"2004ed50-4110-4eb9-b8e6-8516a188544f\" (UID: \"2004ed50-4110-4eb9-b8e6-8516a188544f\") "
Sep 18 19:12:44.261462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.261394    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2004ed50-4110-4eb9-b8e6-8516a188544f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:44.263111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.262659    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:44.263111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.262812    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:44.263111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.263002    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:44.266916 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.263358    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:44.266916 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.263814    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:44.267709 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.267674    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:12:44.267895 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.267859    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-kube-api-access-bs68h" (OuterVolumeSpecName: "kube-api-access-bs68h") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "kube-api-access-bs68h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:44.268106 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.268084    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2004ed50-4110-4eb9-b8e6-8516a188544f" (UID: "2004ed50-4110-4eb9-b8e6-8516a188544f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:12:44.286878 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.286854    2568 scope.go:117] "RemoveContainer" containerID="1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"
Sep 18 19:12:44.297475 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.297456    2568 scope.go:117] "RemoveContainer" containerID="adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3"
Sep 18 19:12:44.297865 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:12:44.297843    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3\": container with ID starting with adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3 not found: ID does not exist" containerID="adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3"
Sep 18 19:12:44.297937 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.297874    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3"} err="failed to get container status \"adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3\": rpc error: code = NotFound desc = could not find container \"adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3\": container with ID starting with adbcb654e8c03cb39467ea721d6868aab0d34ff5393cf7ad29a241b7080369c3 not found: ID does not exist"
Sep 18 19:12:44.297937 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.297893    2568 scope.go:117] "RemoveContainer" containerID="1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"
Sep 18 19:12:44.298176 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:12:44.298141    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6\": container with ID starting with 1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6 not found: ID does not exist" containerID="1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"
Sep 18 19:12:44.298236 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.298185    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6"} err="failed to get container status \"1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6\": rpc error: code = NotFound desc = could not find container \"1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6\": container with ID starting with 1abb69e0766ad57abdf69a50a674392a54d886937c0008d78a6cdec7e70056f6 not found: ID does not exist"
Sep 18 19:12:44.361532 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361482    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361532 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361527    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361532 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361540    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361551    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs68h\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-kube-api-access-bs68h\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361564    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361576    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361587    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2004ed50-4110-4eb9-b8e6-8516a188544f-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361598    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.361758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.361609    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2004ed50-4110-4eb9-b8e6-8516a188544f-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:44.597843 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.597810    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"]
Sep 18 19:12:44.617645 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:44.617608    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1d6196-openshift-default-64596f7bbb-sdv8w"]
Sep 18 19:12:45.055769 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:45.055734    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" path="/var/lib/kubelet/pods/2004ed50-4110-4eb9-b8e6-8516a188544f/volumes"
Sep 18 19:12:55.155614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.155509    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"]
Sep 18 19:12:55.156085 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.155814    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz" podUID="aff1d110-ac96-4a89-bc91-c99527600b1f" containerName="payload-processing" containerID="cri-o://3a86211ed12ff0d25cbb9a32059f48031a1abb19bd3cbed8261615474245d489" gracePeriod=30
Sep 18 19:12:55.168734 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.168705    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"]
Sep 18 19:12:55.169111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.169081    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2" podUID="d043091e-4018-4a30-85b9-bc1f880a2a75" containerName="payload-pre-processing" containerID="cri-o://5a2daebe7a85a563e3dfe2afbe49238efd2718315ef66ee60b3baaa9847f8937" gracePeriod=30
Sep 18 19:12:55.325073 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.324990    2568 generic.go:356] "Generic (PLEG): container finished" podID="d043091e-4018-4a30-85b9-bc1f880a2a75" containerID="5a2daebe7a85a563e3dfe2afbe49238efd2718315ef66ee60b3baaa9847f8937" exitCode=0
Sep 18 19:12:55.325073 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.325056    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2" event={"ID":"d043091e-4018-4a30-85b9-bc1f880a2a75","Type":"ContainerDied","Data":"5a2daebe7a85a563e3dfe2afbe49238efd2718315ef66ee60b3baaa9847f8937"}
Sep 18 19:12:55.327090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.327054    2568 generic.go:356] "Generic (PLEG): container finished" podID="aff1d110-ac96-4a89-bc91-c99527600b1f" containerID="3a86211ed12ff0d25cbb9a32059f48031a1abb19bd3cbed8261615474245d489" exitCode=0
Sep 18 19:12:55.327252 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.327112    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz" event={"ID":"aff1d110-ac96-4a89-bc91-c99527600b1f","Type":"ContainerDied","Data":"3a86211ed12ff0d25cbb9a32059f48031a1abb19bd3cbed8261615474245d489"}
Sep 18 19:12:55.366139 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.366096    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"]
Sep 18 19:12:55.374786 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.374721    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-60d8bd-8nlxn"]
Sep 18 19:12:55.437336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.437312    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:12:55.477908 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.477882    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:12:55.585830 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.585796    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn285\" (UniqueName: \"kubernetes.io/projected/aff1d110-ac96-4a89-bc91-c99527600b1f-kube-api-access-zn285\") pod \"aff1d110-ac96-4a89-bc91-c99527600b1f\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") "
Sep 18 19:12:55.585997 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.585903    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v99hg\" (UniqueName: \"kubernetes.io/projected/d043091e-4018-4a30-85b9-bc1f880a2a75-kube-api-access-v99hg\") pod \"d043091e-4018-4a30-85b9-bc1f880a2a75\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") "
Sep 18 19:12:55.585997 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.585962    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d043091e-4018-4a30-85b9-bc1f880a2a75-plugins-config-volume\") pod \"d043091e-4018-4a30-85b9-bc1f880a2a75\" (UID: \"d043091e-4018-4a30-85b9-bc1f880a2a75\") "
Sep 18 19:12:55.585997 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.585994    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff1d110-ac96-4a89-bc91-c99527600b1f-plugins-config-volume\") pod \"aff1d110-ac96-4a89-bc91-c99527600b1f\" (UID: \"aff1d110-ac96-4a89-bc91-c99527600b1f\") "
Sep 18 19:12:55.586383 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.586355    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d043091e-4018-4a30-85b9-bc1f880a2a75-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d043091e-4018-4a30-85b9-bc1f880a2a75" (UID: "d043091e-4018-4a30-85b9-bc1f880a2a75"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:55.586500 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.586383    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff1d110-ac96-4a89-bc91-c99527600b1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aff1d110-ac96-4a89-bc91-c99527600b1f" (UID: "aff1d110-ac96-4a89-bc91-c99527600b1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:12:55.587905 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.587888    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff1d110-ac96-4a89-bc91-c99527600b1f-kube-api-access-zn285" (OuterVolumeSpecName: "kube-api-access-zn285") pod "aff1d110-ac96-4a89-bc91-c99527600b1f" (UID: "aff1d110-ac96-4a89-bc91-c99527600b1f"). InnerVolumeSpecName "kube-api-access-zn285". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:55.588065 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.588053    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d043091e-4018-4a30-85b9-bc1f880a2a75-kube-api-access-v99hg" (OuterVolumeSpecName: "kube-api-access-v99hg") pod "d043091e-4018-4a30-85b9-bc1f880a2a75" (UID: "d043091e-4018-4a30-85b9-bc1f880a2a75"). InnerVolumeSpecName "kube-api-access-v99hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:12:55.687255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.687172    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v99hg\" (UniqueName: \"kubernetes.io/projected/d043091e-4018-4a30-85b9-bc1f880a2a75-kube-api-access-v99hg\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:55.687255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.687198    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d043091e-4018-4a30-85b9-bc1f880a2a75-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:55.687255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.687207    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff1d110-ac96-4a89-bc91-c99527600b1f-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:55.687255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:55.687227    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn285\" (UniqueName: \"kubernetes.io/projected/aff1d110-ac96-4a89-bc91-c99527600b1f-kube-api-access-zn285\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:12:56.334035 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.334007    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"
Sep 18 19:12:56.334485 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.334043    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2" event={"ID":"d043091e-4018-4a30-85b9-bc1f880a2a75","Type":"ContainerDied","Data":"93029a8bdc9ecf625a8d7aa38a4538a34e107da2506c584581dbaa0309ede41e"}
Sep 18 19:12:56.334485 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.334104    2568 scope.go:117] "RemoveContainer" containerID="5a2daebe7a85a563e3dfe2afbe49238efd2718315ef66ee60b3baaa9847f8937"
Sep 18 19:12:56.335579 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.335553    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz" event={"ID":"aff1d110-ac96-4a89-bc91-c99527600b1f","Type":"ContainerDied","Data":"f3316c94454f9b3c1a06186e80e677a53d401c42d642a66bff30321084a494f7"}
Sep 18 19:12:56.335671 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.335576    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"
Sep 18 19:12:56.345381 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.345364    2568 scope.go:117] "RemoveContainer" containerID="3a86211ed12ff0d25cbb9a32059f48031a1abb19bd3cbed8261615474245d489"
Sep 18 19:12:56.367071 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.367042    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"]
Sep 18 19:12:56.374390 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.374370    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-60d8bd-68f67f5566-6vfr2"]
Sep 18 19:12:56.392958 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.392935    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"]
Sep 18 19:12:56.405361 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:56.405336    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-60d8bd-78cb989877-tq8xz"]
Sep 18 19:12:57.055076 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:57.055044    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b9290a3-81cf-417f-9486-14ab965cdb75" path="/var/lib/kubelet/pods/0b9290a3-81cf-417f-9486-14ab965cdb75/volumes"
Sep 18 19:12:57.055447 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:57.055433    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff1d110-ac96-4a89-bc91-c99527600b1f" path="/var/lib/kubelet/pods/aff1d110-ac96-4a89-bc91-c99527600b1f/volumes"
Sep 18 19:12:57.055783 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:57.055771    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d043091e-4018-4a30-85b9-bc1f880a2a75" path="/var/lib/kubelet/pods/d043091e-4018-4a30-85b9-bc1f880a2a75/volumes"
Sep 18 19:12:59.191091 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191057    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"]
Sep 18 19:12:59.191566 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191551    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191567    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191578    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d043091e-4018-4a30-85b9-bc1f880a2a75" containerName="payload-pre-processing"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191584    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d043091e-4018-4a30-85b9-bc1f880a2a75" containerName="payload-pre-processing"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191606    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff1d110-ac96-4a89-bc91-c99527600b1f" containerName="payload-processing"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191612    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff1d110-ac96-4a89-bc91-c99527600b1f" containerName="payload-processing"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191618    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy"
Sep 18 19:12:59.191624 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191624    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy"
Sep 18 19:12:59.191870 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191631    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b9290a3-81cf-417f-9486-14ab965cdb75" containerName="revoke-keys"
Sep 18 19:12:59.191870 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191636    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b9290a3-81cf-417f-9486-14ab965cdb75" containerName="revoke-keys"
Sep 18 19:12:59.191870 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191707    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b9290a3-81cf-417f-9486-14ab965cdb75" containerName="revoke-keys"
Sep 18 19:12:59.191870 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191715    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff1d110-ac96-4a89-bc91-c99527600b1f" containerName="payload-processing"
Sep 18 19:12:59.191870 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191721    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy"
Sep 18 19:12:59.191870 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.191731    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d043091e-4018-4a30-85b9-bc1f880a2a75" containerName="payload-pre-processing"
Sep 18 19:12:59.196160 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.196117    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:12:59.199263 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.199236    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:12:59.199413 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.199391    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:12:59.225542 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.225493    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"]
Sep 18 19:12:59.325927 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.325881    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f4d4e97-e356-481e-af99-99a98376a031-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx\" (UID: \"8f4d4e97-e356-481e-af99-99a98376a031\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:12:59.426708 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.426673    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f4d4e97-e356-481e-af99-99a98376a031-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx\" (UID: \"8f4d4e97-e356-481e-af99-99a98376a031\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:12:59.427269 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.427250    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f4d4e97-e356-481e-af99-99a98376a031-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx\" (UID: \"8f4d4e97-e356-481e-af99-99a98376a031\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:12:59.506068 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.506032    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:12:59.809985 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:12:59.809884    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"]
Sep 18 19:12:59.819727 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:12:59.819685    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4d4e97_e356_481e_af99_99a98376a031.slice/crio-1f84975b21eeb5b102c7ade44dff62a70396ec643936238ef43e9131b9446613 WatchSource:0}: Error finding container 1f84975b21eeb5b102c7ade44dff62a70396ec643936238ef43e9131b9446613: Status 404 returned error can't find the container with id 1f84975b21eeb5b102c7ade44dff62a70396ec643936238ef43e9131b9446613
Sep 18 19:13:00.372425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:00.372391    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx" event={"ID":"8f4d4e97-e356-481e-af99-99a98376a031","Type":"ContainerStarted","Data":"a3f047950f7b318768494a30a8bcf47917ac1ebc9fb34616dbd5a6d2f409444d"}
Sep 18 19:13:00.372818 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:00.372446    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx" event={"ID":"8f4d4e97-e356-481e-af99-99a98376a031","Type":"ContainerStarted","Data":"1f84975b21eeb5b102c7ade44dff62a70396ec643936238ef43e9131b9446613"}
Sep 18 19:13:00.407464 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:00.407366    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx" podStartSLOduration=1.407350087 podStartE2EDuration="1.407350087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:13:00.406990609 +0000 UTC m=+1709.845250806" watchObservedRunningTime="2026-09-18 19:13:00.407350087 +0000 UTC m=+1709.845610290"
Sep 18 19:13:01.379783 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:01.379749    2568 generic.go:356] "Generic (PLEG): container finished" podID="8f4d4e97-e356-481e-af99-99a98376a031" containerID="a3f047950f7b318768494a30a8bcf47917ac1ebc9fb34616dbd5a6d2f409444d" exitCode=0
Sep 18 19:13:01.380157 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:01.379798    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx" event={"ID":"8f4d4e97-e356-481e-af99-99a98376a031","Type":"ContainerDied","Data":"a3f047950f7b318768494a30a8bcf47917ac1ebc9fb34616dbd5a6d2f409444d"}
Sep 18 19:13:02.530568 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:02.530537    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:13:02.663389 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:02.663311    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f4d4e97-e356-481e-af99-99a98376a031-maas-api-service-ca\") pod \"8f4d4e97-e356-481e-af99-99a98376a031\" (UID: \"8f4d4e97-e356-481e-af99-99a98376a031\") "
Sep 18 19:13:02.663691 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:02.663669    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f4d4e97-e356-481e-af99-99a98376a031-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f4d4e97-e356-481e-af99-99a98376a031" (UID: "8f4d4e97-e356-481e-af99-99a98376a031"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:02.764385 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:02.764350    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f4d4e97-e356-481e-af99-99a98376a031-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:03.393396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:03.393310    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx" event={"ID":"8f4d4e97-e356-481e-af99-99a98376a031","Type":"ContainerDied","Data":"1f84975b21eeb5b102c7ade44dff62a70396ec643936238ef43e9131b9446613"}
Sep 18 19:13:03.393396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:03.393330    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"
Sep 18 19:13:03.393396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:03.393356    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f84975b21eeb5b102c7ade44dff62a70396ec643936238ef43e9131b9446613"
Sep 18 19:13:04.892575 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.892538    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"]
Sep 18 19:13:04.893028 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.893014    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f4d4e97-e356-481e-af99-99a98376a031" containerName="revoke-keys"
Sep 18 19:13:04.893028 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.893029    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f4d4e97-e356-481e-af99-99a98376a031" containerName="revoke-keys"
Sep 18 19:13:04.893134 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.893097    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f4d4e97-e356-481e-af99-99a98376a031" containerName="revoke-keys"
Sep 18 19:13:04.893134 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.893104    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2004ed50-4110-4eb9-b8e6-8516a188544f" containerName="istio-proxy"
Sep 18 19:13:04.896193 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.896176    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:04.900173 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.900154    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:13:04.900284 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.900207    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:13:04.916590 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.916565    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"]
Sep 18 19:13:04.980185 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:04.980147    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a62e275-c6cc-4bf6-befa-a59150372936-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc\" (UID: \"8a62e275-c6cc-4bf6-befa-a59150372936\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:05.080733 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:05.080695    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a62e275-c6cc-4bf6-befa-a59150372936-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc\" (UID: \"8a62e275-c6cc-4bf6-befa-a59150372936\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:05.081356 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:05.081339    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a62e275-c6cc-4bf6-befa-a59150372936-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc\" (UID: \"8a62e275-c6cc-4bf6-befa-a59150372936\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:05.206293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:05.206199    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:05.481282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:05.481256    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"]
Sep 18 19:13:05.482838 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:13:05.482802    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a62e275_c6cc_4bf6_befa_a59150372936.slice/crio-655e3e333849f21130e8f59506a275f0b3fb4f51f4342486c1125da7cb1c196d WatchSource:0}: Error finding container 655e3e333849f21130e8f59506a275f0b3fb4f51f4342486c1125da7cb1c196d: Status 404 returned error can't find the container with id 655e3e333849f21130e8f59506a275f0b3fb4f51f4342486c1125da7cb1c196d
Sep 18 19:13:06.412089 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:06.412056    2568 generic.go:356] "Generic (PLEG): container finished" podID="8a62e275-c6cc-4bf6-befa-a59150372936" containerID="c24acc6fc672373f56292ea2c76ac4f0830f9b4031bb489461d6b09a7cb54ffd" exitCode=0
Sep 18 19:13:06.412830 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:06.412148    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc" event={"ID":"8a62e275-c6cc-4bf6-befa-a59150372936","Type":"ContainerDied","Data":"c24acc6fc672373f56292ea2c76ac4f0830f9b4031bb489461d6b09a7cb54ffd"}
Sep 18 19:13:06.412830 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:06.412217    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc" event={"ID":"8a62e275-c6cc-4bf6-befa-a59150372936","Type":"ContainerStarted","Data":"655e3e333849f21130e8f59506a275f0b3fb4f51f4342486c1125da7cb1c196d"}
Sep 18 19:13:07.567765 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:07.567736    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:07.603791 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:07.603762    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a62e275-c6cc-4bf6-befa-a59150372936-maas-api-service-ca\") pod \"8a62e275-c6cc-4bf6-befa-a59150372936\" (UID: \"8a62e275-c6cc-4bf6-befa-a59150372936\") "
Sep 18 19:13:07.604185 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:07.604160    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a62e275-c6cc-4bf6-befa-a59150372936-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8a62e275-c6cc-4bf6-befa-a59150372936" (UID: "8a62e275-c6cc-4bf6-befa-a59150372936"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:07.705200 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:07.705113    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a62e275-c6cc-4bf6-befa-a59150372936-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:08.426138 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:08.426107    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"
Sep 18 19:13:08.426300 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:08.426132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc" event={"ID":"8a62e275-c6cc-4bf6-befa-a59150372936","Type":"ContainerDied","Data":"655e3e333849f21130e8f59506a275f0b3fb4f51f4342486c1125da7cb1c196d"}
Sep 18 19:13:08.426300 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:08.426176    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="655e3e333849f21130e8f59506a275f0b3fb4f51f4342486c1125da7cb1c196d"
Sep 18 19:13:14.875775 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:14.875740    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"]
Sep 18 19:13:14.886440 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:14.886409    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-942883-29829300-dnhwh"]
Sep 18 19:13:14.965629 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:14.965595    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"]
Sep 18 19:13:14.965868 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:14.965842    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll" podUID="20e186dc-5701-4850-9d39-c07b61f199d3" containerName="payload-pre-processing" containerID="cri-o://95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f" gracePeriod=30
Sep 18 19:13:15.054613 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.054580    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5a53f7b-9b25-4866-94d5-3316689d7d73" path="/var/lib/kubelet/pods/a5a53f7b-9b25-4866-94d5-3316689d7d73/volumes"
Sep 18 19:13:15.238059 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.238036    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 19:13:15.286763 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.286727    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20e186dc-5701-4850-9d39-c07b61f199d3-plugins-config-volume\") pod \"20e186dc-5701-4850-9d39-c07b61f199d3\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") "
Sep 18 19:13:15.286952 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.286775    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m58gl\" (UniqueName: \"kubernetes.io/projected/20e186dc-5701-4850-9d39-c07b61f199d3-kube-api-access-m58gl\") pod \"20e186dc-5701-4850-9d39-c07b61f199d3\" (UID: \"20e186dc-5701-4850-9d39-c07b61f199d3\") "
Sep 18 19:13:15.287103 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.287069    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e186dc-5701-4850-9d39-c07b61f199d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20e186dc-5701-4850-9d39-c07b61f199d3" (UID: "20e186dc-5701-4850-9d39-c07b61f199d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:15.288775 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.288752    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e186dc-5701-4850-9d39-c07b61f199d3-kube-api-access-m58gl" (OuterVolumeSpecName: "kube-api-access-m58gl") pod "20e186dc-5701-4850-9d39-c07b61f199d3" (UID: "20e186dc-5701-4850-9d39-c07b61f199d3"). InnerVolumeSpecName "kube-api-access-m58gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:15.387847 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.387766    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20e186dc-5701-4850-9d39-c07b61f199d3-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.387847 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.387792    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m58gl\" (UniqueName: \"kubernetes.io/projected/20e186dc-5701-4850-9d39-c07b61f199d3-kube-api-access-m58gl\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.424903 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.424866    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"]
Sep 18 19:13:15.425135 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.425110    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8" podUID="3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" containerName="payload-processing" containerID="cri-o://4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b" gracePeriod=30
Sep 18 19:13:15.442973 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.442944    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"]
Sep 18 19:13:15.443210 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.443167    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws" podUID="5e2bae64-e619-40db-967a-0ac0fe1216a0" containerName="payload-pre-processing" containerID="cri-o://86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c" gracePeriod=30
Sep 18 19:13:15.471298 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.471253    2568 generic.go:356] "Generic (PLEG): container finished" podID="20e186dc-5701-4850-9d39-c07b61f199d3" containerID="95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f" exitCode=0
Sep 18 19:13:15.471480 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.471314    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"
Sep 18 19:13:15.471480 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.471430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll" event={"ID":"20e186dc-5701-4850-9d39-c07b61f199d3","Type":"ContainerDied","Data":"95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f"}
Sep 18 19:13:15.471622 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.471481    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll" event={"ID":"20e186dc-5701-4850-9d39-c07b61f199d3","Type":"ContainerDied","Data":"3926a6433e1add1097cfb742a8f8a14ab0064ac906aec931bc91f68f675a315a"}
Sep 18 19:13:15.471622 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.471542    2568 scope.go:117] "RemoveContainer" containerID="95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f"
Sep 18 19:13:15.490260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.490238    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"]
Sep 18 19:13:15.499063 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.499035    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-942883-vxjrx"]
Sep 18 19:13:15.555864 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.555839    2568 scope.go:117] "RemoveContainer" containerID="95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f"
Sep 18 19:13:15.556253 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:13:15.556223    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f\": container with ID starting with 95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f not found: ID does not exist" containerID="95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f"
Sep 18 19:13:15.556317 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.556267    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f"} err="failed to get container status \"95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f\": rpc error: code = NotFound desc = could not find container \"95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f\": container with ID starting with 95ae5cc523f1dd0acce4a65343d5e710c545ef7db966e3abe70400770003625f not found: ID does not exist"
Sep 18 19:13:15.590849 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.590792    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"]
Sep 18 19:13:15.603568 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.602330    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-942883-7b8bb69f78-prnll"]
Sep 18 19:13:15.717957 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.717939    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:13:15.723311 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.723291    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:13:15.793538 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.792720    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmzng\" (UniqueName: \"kubernetes.io/projected/5e2bae64-e619-40db-967a-0ac0fe1216a0-kube-api-access-mmzng\") pod \"5e2bae64-e619-40db-967a-0ac0fe1216a0\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") "
Sep 18 19:13:15.793538 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.792787    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-plugins-config-volume\") pod \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") "
Sep 18 19:13:15.793538 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.792904    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2bae64-e619-40db-967a-0ac0fe1216a0-plugins-config-volume\") pod \"5e2bae64-e619-40db-967a-0ac0fe1216a0\" (UID: \"5e2bae64-e619-40db-967a-0ac0fe1216a0\") "
Sep 18 19:13:15.793538 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.792972    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppg9z\" (UniqueName: \"kubernetes.io/projected/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-kube-api-access-ppg9z\") pod \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\" (UID: \"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a\") "
Sep 18 19:13:15.794294 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.794161    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" (UID: "3522ed84-bc0d-4a8b-a5b6-665847aa6a4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:15.794294 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.794214    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e2bae64-e619-40db-967a-0ac0fe1216a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e2bae64-e619-40db-967a-0ac0fe1216a0" (UID: "5e2bae64-e619-40db-967a-0ac0fe1216a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:15.797454 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.797373    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-kube-api-access-ppg9z" (OuterVolumeSpecName: "kube-api-access-ppg9z") pod "3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" (UID: "3522ed84-bc0d-4a8b-a5b6-665847aa6a4a"). InnerVolumeSpecName "kube-api-access-ppg9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:15.798904 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.798875    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2bae64-e619-40db-967a-0ac0fe1216a0-kube-api-access-mmzng" (OuterVolumeSpecName: "kube-api-access-mmzng") pod "5e2bae64-e619-40db-967a-0ac0fe1216a0" (UID: "5e2bae64-e619-40db-967a-0ac0fe1216a0"). InnerVolumeSpecName "kube-api-access-mmzng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:15.894299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.894247    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmzng\" (UniqueName: \"kubernetes.io/projected/5e2bae64-e619-40db-967a-0ac0fe1216a0-kube-api-access-mmzng\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.894299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.894294    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.894299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.894304    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e2bae64-e619-40db-967a-0ac0fe1216a0-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:15.894299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:15.894315    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppg9z\" (UniqueName: \"kubernetes.io/projected/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a-kube-api-access-ppg9z\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:16.212444 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.212407    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"]
Sep 18 19:13:16.214588 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.214566    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3b5822-jm4nc"]
Sep 18 19:13:16.478873 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.478843    2568 generic.go:356] "Generic (PLEG): container finished" podID="5e2bae64-e619-40db-967a-0ac0fe1216a0" containerID="86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c" exitCode=0
Sep 18 19:13:16.479078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.478898    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"
Sep 18 19:13:16.479078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.478932    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws" event={"ID":"5e2bae64-e619-40db-967a-0ac0fe1216a0","Type":"ContainerDied","Data":"86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c"}
Sep 18 19:13:16.479078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.478985    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws" event={"ID":"5e2bae64-e619-40db-967a-0ac0fe1216a0","Type":"ContainerDied","Data":"2eea2614262e2535defeb7a42f147d5f7044e4117c6162ccca2cc6c138340cdf"}
Sep 18 19:13:16.479078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.479011    2568 scope.go:117] "RemoveContainer" containerID="86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c"
Sep 18 19:13:16.480448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.480425    2568 generic.go:356] "Generic (PLEG): container finished" podID="3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" containerID="4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b" exitCode=0
Sep 18 19:13:16.480545 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.480462    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"
Sep 18 19:13:16.480545 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.480508    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8" event={"ID":"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a","Type":"ContainerDied","Data":"4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b"}
Sep 18 19:13:16.480669 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.480564    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8" event={"ID":"3522ed84-bc0d-4a8b-a5b6-665847aa6a4a","Type":"ContainerDied","Data":"7d97b147f0f51c46cfef53832edbc5a4318ac6def0f2ba90b63636673b2612ba"}
Sep 18 19:13:16.489377 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.489360    2568 scope.go:117] "RemoveContainer" containerID="86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c"
Sep 18 19:13:16.489715 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:13:16.489690    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c\": container with ID starting with 86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c not found: ID does not exist" containerID="86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c"
Sep 18 19:13:16.489828 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.489723    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c"} err="failed to get container status \"86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c\": rpc error: code = NotFound desc = could not find container \"86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c\": container with ID starting with 86b4e8dec542b1055899bade5d4fc603292fb1c46f82cd59f3fd46c57edd775c not found: ID does not exist"
Sep 18 19:13:16.489828 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.489745    2568 scope.go:117] "RemoveContainer" containerID="4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b"
Sep 18 19:13:16.498030 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.498012    2568 scope.go:117] "RemoveContainer" containerID="4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b"
Sep 18 19:13:16.498274 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:13:16.498256    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b\": container with ID starting with 4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b not found: ID does not exist" containerID="4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b"
Sep 18 19:13:16.498321 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.498281    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b"} err="failed to get container status \"4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b\": rpc error: code = NotFound desc = could not find container \"4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b\": container with ID starting with 4c668b1a1811ae4d195e808877725cec8eef43332f44d9aff437584d01df337b not found: ID does not exist"
Sep 18 19:13:16.553662 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.553626    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"]
Sep 18 19:13:16.572753 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.572723    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b5822-768946f969-82bj8"]
Sep 18 19:13:16.603274 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.603242    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"]
Sep 18 19:13:16.618772 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:16.618740    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b5822-7fb57c7f8-wlbws"]
Sep 18 19:13:17.055153 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:17.055124    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e186dc-5701-4850-9d39-c07b61f199d3" path="/var/lib/kubelet/pods/20e186dc-5701-4850-9d39-c07b61f199d3/volumes"
Sep 18 19:13:17.055558 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:17.055544    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" path="/var/lib/kubelet/pods/3522ed84-bc0d-4a8b-a5b6-665847aa6a4a/volumes"
Sep 18 19:13:17.055875 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:17.055864    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e2bae64-e619-40db-967a-0ac0fe1216a0" path="/var/lib/kubelet/pods/5e2bae64-e619-40db-967a-0ac0fe1216a0/volumes"
Sep 18 19:13:17.056149 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:17.056139    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a62e275-c6cc-4bf6-befa-a59150372936" path="/var/lib/kubelet/pods/8a62e275-c6cc-4bf6-befa-a59150372936/volumes"
Sep 18 19:13:17.056759 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:17.056746    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f4d4e97-e356-481e-af99-99a98376a031" path="/var/lib/kubelet/pods/8f4d4e97-e356-481e-af99-99a98376a031/volumes"
Sep 18 19:13:24.961841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:24.961806    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"]
Sep 18 19:13:24.962295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:24.962015    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy" containerID="cri-o://e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974" gracePeriod=30
Sep 18 19:13:26.121878 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.121827    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"]
Sep 18 19:13:26.122328 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.122074    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy" containerID="cri-o://568f4e22fe22bc1d495712c75af74f52ae755d0acea149c831970dbf29fe4afd" gracePeriod=30
Sep 18 19:13:26.467109 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467075    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"]
Sep 18 19:13:26.467564 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467550    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2bae64-e619-40db-967a-0ac0fe1216a0" containerName="payload-pre-processing"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467565    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2bae64-e619-40db-967a-0ac0fe1216a0" containerName="payload-pre-processing"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467581    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e186dc-5701-4850-9d39-c07b61f199d3" containerName="payload-pre-processing"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467586    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e186dc-5701-4850-9d39-c07b61f199d3" containerName="payload-pre-processing"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467594    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" containerName="payload-processing"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467599    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" containerName="payload-processing"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467606    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a62e275-c6cc-4bf6-befa-a59150372936" containerName="revoke-keys"
Sep 18 19:13:26.467614 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467611    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a62e275-c6cc-4bf6-befa-a59150372936" containerName="revoke-keys"
Sep 18 19:13:26.467829 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467672    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a62e275-c6cc-4bf6-befa-a59150372936" containerName="revoke-keys"
Sep 18 19:13:26.467829 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467680    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e186dc-5701-4850-9d39-c07b61f199d3" containerName="payload-pre-processing"
Sep 18 19:13:26.467829 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467688    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e2bae64-e619-40db-967a-0ac0fe1216a0" containerName="payload-pre-processing"
Sep 18 19:13:26.467829 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.467697    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3522ed84-bc0d-4a8b-a5b6-665847aa6a4a" containerName="payload-processing"
Sep 18 19:13:26.472437 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.472418    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:26.476704 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.476685    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:13:26.476704 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.476693    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:13:26.490426 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.490394    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"]
Sep 18 19:13:26.602752 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.602715    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/553fe217-88b7-41fb-bb78-c0e6371d9b78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss\" (UID: \"553fe217-88b7-41fb-bb78-c0e6371d9b78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:26.703841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.703790    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/553fe217-88b7-41fb-bb78-c0e6371d9b78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss\" (UID: \"553fe217-88b7-41fb-bb78-c0e6371d9b78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:26.704400 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.704380    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/553fe217-88b7-41fb-bb78-c0e6371d9b78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss\" (UID: \"553fe217-88b7-41fb-bb78-c0e6371d9b78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:26.782875 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:26.782784    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:27.077049 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:13:27.075724    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553fe217_88b7_41fb_bb78_c0e6371d9b78.slice/crio-31ba5c0a0b4f9d2f0991268d39ff09687f318b0a7cf8266aca3215b3e7412b45 WatchSource:0}: Error finding container 31ba5c0a0b4f9d2f0991268d39ff09687f318b0a7cf8266aca3215b3e7412b45: Status 404 returned error can't find the container with id 31ba5c0a0b4f9d2f0991268d39ff09687f318b0a7cf8266aca3215b3e7412b45
Sep 18 19:13:27.078055 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:27.078022    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"]
Sep 18 19:13:27.549899 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:27.549861    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss" event={"ID":"553fe217-88b7-41fb-bb78-c0e6371d9b78","Type":"ContainerStarted","Data":"9bbb9302f05d904ec07207fe476c08db62d4f629e9e84c735d3fa2e693e9bbd3"}
Sep 18 19:13:27.550276 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:27.549912    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss" event={"ID":"553fe217-88b7-41fb-bb78-c0e6371d9b78","Type":"ContainerStarted","Data":"31ba5c0a0b4f9d2f0991268d39ff09687f318b0a7cf8266aca3215b3e7412b45"}
Sep 18 19:13:27.574124 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:27.574075    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss" podStartSLOduration=1.574057416 podStartE2EDuration="1.574057416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:13:27.572207061 +0000 UTC m=+1737.010467267" watchObservedRunningTime="2026-09-18 19:13:27.574057416 +0000 UTC m=+1737.012317619"
Sep 18 19:13:28.557384 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:28.557349    2568 generic.go:356] "Generic (PLEG): container finished" podID="553fe217-88b7-41fb-bb78-c0e6371d9b78" containerID="9bbb9302f05d904ec07207fe476c08db62d4f629e9e84c735d3fa2e693e9bbd3" exitCode=0
Sep 18 19:13:28.557784 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:28.557438    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss" event={"ID":"553fe217-88b7-41fb-bb78-c0e6371d9b78","Type":"ContainerDied","Data":"9bbb9302f05d904ec07207fe476c08db62d4f629e9e84c735d3fa2e693e9bbd3"}
Sep 18 19:13:29.703127 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:29.703102    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:29.837126 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:29.837042    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/553fe217-88b7-41fb-bb78-c0e6371d9b78-maas-api-service-ca\") pod \"553fe217-88b7-41fb-bb78-c0e6371d9b78\" (UID: \"553fe217-88b7-41fb-bb78-c0e6371d9b78\") "
Sep 18 19:13:29.837395 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:29.837371    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/553fe217-88b7-41fb-bb78-c0e6371d9b78-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "553fe217-88b7-41fb-bb78-c0e6371d9b78" (UID: "553fe217-88b7-41fb-bb78-c0e6371d9b78"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:29.938107 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:29.938071    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/553fe217-88b7-41fb-bb78-c0e6371d9b78-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.224346 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.224325    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 19:13:30.343735 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343692    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-data\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343735 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343752    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-certs\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343777    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-token\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343844    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-podinfo\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343861    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-envoy\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343892    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-credential-socket\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343926    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smk4c\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-kube-api-access-smk4c\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.343994 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.343987    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-socket\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.344312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344029    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e88335c-687d-4816-b5ca-1a42e08247d0-istiod-ca-cert\") pod \"4e88335c-687d-4816-b5ca-1a42e08247d0\" (UID: \"4e88335c-687d-4816-b5ca-1a42e08247d0\") "
Sep 18 19:13:30.344312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344013    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:30.344312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344072    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:30.344312 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344275    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:30.344488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344389    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:30.344488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344400    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.344488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344439    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.344488 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344456    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.344666 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.344492    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e88335c-687d-4816-b5ca-1a42e08247d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:30.346088 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.346060    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:13:30.346287 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.346265    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:30.346480 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.346459    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-kube-api-access-smk4c" (OuterVolumeSpecName: "kube-api-access-smk4c") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "kube-api-access-smk4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:30.346554 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.346467    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e88335c-687d-4816-b5ca-1a42e08247d0" (UID: "4e88335c-687d-4816-b5ca-1a42e08247d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:30.445248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.445155    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.445248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.445189    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.445248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.445202    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e88335c-687d-4816-b5ca-1a42e08247d0-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.445248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.445215    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smk4c\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-kube-api-access-smk4c\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.445248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.445228    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e88335c-687d-4816-b5ca-1a42e08247d0-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.445248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.445240    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e88335c-687d-4816-b5ca-1a42e08247d0-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:30.571577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.571541    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerID="e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974" exitCode=0
Sep 18 19:13:30.571749 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.571599    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"
Sep 18 19:13:30.571749 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.571633    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" event={"ID":"4e88335c-687d-4816-b5ca-1a42e08247d0","Type":"ContainerDied","Data":"e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974"}
Sep 18 19:13:30.571749 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.571688    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz" event={"ID":"4e88335c-687d-4816-b5ca-1a42e08247d0","Type":"ContainerDied","Data":"389bed87fbbf30dd19c34a4408fa81d4a6461495ce0ecaad7a9619dd5b9b2ada"}
Sep 18 19:13:30.571749 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.571722    2568 scope.go:117] "RemoveContainer" containerID="e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974"
Sep 18 19:13:30.573090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.573017    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"
Sep 18 19:13:30.573090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.573023    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss" event={"ID":"553fe217-88b7-41fb-bb78-c0e6371d9b78","Type":"ContainerDied","Data":"31ba5c0a0b4f9d2f0991268d39ff09687f318b0a7cf8266aca3215b3e7412b45"}
Sep 18 19:13:30.573090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.573068    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31ba5c0a0b4f9d2f0991268d39ff09687f318b0a7cf8266aca3215b3e7412b45"
Sep 18 19:13:30.581457 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.581441    2568 scope.go:117] "RemoveContainer" containerID="6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"
Sep 18 19:13:30.589891 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.589875    2568 scope.go:117] "RemoveContainer" containerID="e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974"
Sep 18 19:13:30.590176 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:13:30.590157    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974\": container with ID starting with e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974 not found: ID does not exist" containerID="e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974"
Sep 18 19:13:30.590254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.590189    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974"} err="failed to get container status \"e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974\": rpc error: code = NotFound desc = could not find container \"e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974\": container with ID starting with e0b4a0d6df1bf10a366937c0b42d5e399885c025ef9b861965593dc23ceac974 not found: ID does not exist"
Sep 18 19:13:30.590254 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.590215    2568 scope.go:117] "RemoveContainer" containerID="6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"
Sep 18 19:13:30.590461 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:13:30.590445    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6\": container with ID starting with 6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6 not found: ID does not exist" containerID="6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"
Sep 18 19:13:30.590530 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.590469    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6"} err="failed to get container status \"6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6\": rpc error: code = NotFound desc = could not find container \"6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6\": container with ID starting with 6462332ccd2d27ad9c96472b3b1fa4f20823632f05cff7ab9f2d5e19774881b6 not found: ID does not exist"
Sep 18 19:13:30.606241 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.606207    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"]
Sep 18 19:13:30.617563 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:30.617541    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-942883-openshift-default-56d4fb7cdd-4nxwz"]
Sep 18 19:13:31.057299 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.057256    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" path="/var/lib/kubelet/pods/4e88335c-687d-4816-b5ca-1a42e08247d0/volumes"
Sep 18 19:13:31.372848 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.372815    2568 scope.go:117] "RemoveContainer" containerID="b86812635f8cfd556b91ff95607863f3a25c4efb8d993555e6cac9346934ffdb"
Sep 18 19:13:31.381876 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.381855    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:13:31.382167 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.382149    2568 scope.go:117] "RemoveContainer" containerID="37842cc2b671f31972b6d11b4ede9898f7b6b730516549588f07dc5662d26166"
Sep 18 19:13:31.390245 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.390224    2568 scope.go:117] "RemoveContainer" containerID="5d82bb65ed240b6817a4ba2d47e46de4e98627c40f00f43d35b8d347d4f207de"
Sep 18 19:13:31.398321 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.398304    2568 scope.go:117] "RemoveContainer" containerID="568f4e22fe22bc1d495712c75af74f52ae755d0acea149c831970dbf29fe4afd"
Sep 18 19:13:31.555428 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555396    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-envoy\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555598 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555441    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hgp2\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-kube-api-access-7hgp2\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555598 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555466    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-podinfo\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555598 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555507    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istiod-ca-cert\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555598 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555558    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-credential-socket\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555598 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555581    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-data\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555885 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555657    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-certs\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555885 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555683    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-token\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.555885 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555711    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-socket\") pod \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\" (UID: \"7f6fe400-5f1d-4c17-896d-e1332effb9c4\") "
Sep 18 19:13:31.556037 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555922    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:31.556037 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555982    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:31.556037 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.555991    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:31.556188 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556080    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:31.556188 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556164    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:31.556339 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556317    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.556339 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556340    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.556549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556355    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.556549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556369    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.556549 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.556381    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.557935 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.557908    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:31.558040 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.557944    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-kube-api-access-7hgp2" (OuterVolumeSpecName: "kube-api-access-7hgp2") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "kube-api-access-7hgp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:31.558131 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.558116    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:13:31.558191 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.558119    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f6fe400-5f1d-4c17-896d-e1332effb9c4" (UID: "7f6fe400-5f1d-4c17-896d-e1332effb9c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:13:31.578421 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.578391    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" event={"ID":"7f6fe400-5f1d-4c17-896d-e1332effb9c4","Type":"ContainerDied","Data":"568f4e22fe22bc1d495712c75af74f52ae755d0acea149c831970dbf29fe4afd"}
Sep 18 19:13:31.578547 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.578445    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52" event={"ID":"7f6fe400-5f1d-4c17-896d-e1332effb9c4","Type":"ContainerDied","Data":"f5f58ac951ca0943a93d6127957e50913743043d10574d9d79e3f87d463801af"}
Sep 18 19:13:31.578547 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.578453    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"
Sep 18 19:13:31.643290 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.643203    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"]
Sep 18 19:13:31.647289 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.647264    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3b5822-openshift-default-6c5c444698-lvr52"]
Sep 18 19:13:31.657318 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.657298    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.657318 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.657318    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hgp2\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-kube-api-access-7hgp2\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.657437 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.657327    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:31.657437 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:31.657336    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f6fe400-5f1d-4c17-896d-e1332effb9c4-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:33.055128 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:33.055093    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" path="/var/lib/kubelet/pods/7f6fe400-5f1d-4c17-896d-e1332effb9c4/volumes"
Sep 18 19:13:37.145823 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.145784    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"]
Sep 18 19:13:37.146237 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.146067    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5" podUID="099537b0-bc73-4c11-8905-6424c25e40b0" containerName="payload-processing" containerID="cri-o://70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c" gracePeriod=30
Sep 18 19:13:37.356685 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.356656    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"]
Sep 18 19:13:37.358767 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.358745    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4805a7-8xgss"]
Sep 18 19:13:37.419050 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.419017    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:13:37.516351 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.516320    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099537b0-bc73-4c11-8905-6424c25e40b0-plugins-config-volume\") pod \"099537b0-bc73-4c11-8905-6424c25e40b0\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") "
Sep 18 19:13:37.516351 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.516359    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9rkg\" (UniqueName: \"kubernetes.io/projected/099537b0-bc73-4c11-8905-6424c25e40b0-kube-api-access-d9rkg\") pod \"099537b0-bc73-4c11-8905-6424c25e40b0\" (UID: \"099537b0-bc73-4c11-8905-6424c25e40b0\") "
Sep 18 19:13:37.516733 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.516709    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099537b0-bc73-4c11-8905-6424c25e40b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "099537b0-bc73-4c11-8905-6424c25e40b0" (UID: "099537b0-bc73-4c11-8905-6424c25e40b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:13:37.518623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.518594    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099537b0-bc73-4c11-8905-6424c25e40b0-kube-api-access-d9rkg" (OuterVolumeSpecName: "kube-api-access-d9rkg") pod "099537b0-bc73-4c11-8905-6424c25e40b0" (UID: "099537b0-bc73-4c11-8905-6424c25e40b0"). InnerVolumeSpecName "kube-api-access-d9rkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:13:37.611411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.611378    2568 generic.go:356] "Generic (PLEG): container finished" podID="099537b0-bc73-4c11-8905-6424c25e40b0" containerID="70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c" exitCode=0
Sep 18 19:13:37.611619 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.611421    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"
Sep 18 19:13:37.611619 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.611460    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5" event={"ID":"099537b0-bc73-4c11-8905-6424c25e40b0","Type":"ContainerDied","Data":"70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c"}
Sep 18 19:13:37.611619 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.611536    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5" event={"ID":"099537b0-bc73-4c11-8905-6424c25e40b0","Type":"ContainerDied","Data":"f7f62740fd51e36e347c0411d781eb60e603d00386101973328fadc8481f5d1e"}
Sep 18 19:13:37.611619 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.611561    2568 scope.go:117] "RemoveContainer" containerID="70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c"
Sep 18 19:13:37.617083 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.617059    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/099537b0-bc73-4c11-8905-6424c25e40b0-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:37.617083 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.617080    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9rkg\" (UniqueName: \"kubernetes.io/projected/099537b0-bc73-4c11-8905-6424c25e40b0-kube-api-access-d9rkg\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:13:37.620543 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.620497    2568 scope.go:117] "RemoveContainer" containerID="70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c"
Sep 18 19:13:37.620845 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:13:37.620821    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c\": container with ID starting with 70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c not found: ID does not exist" containerID="70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c"
Sep 18 19:13:37.620925 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.620853    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c"} err="failed to get container status \"70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c\": rpc error: code = NotFound desc = could not find container \"70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c\": container with ID starting with 70c62b37cb60faae839b6d1dce69d471499aa3b28228c8812362cc3fa3320a6c not found: ID does not exist"
Sep 18 19:13:37.650860 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.650788    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"]
Sep 18 19:13:37.661661 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:37.661635    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4805a7-54478f677-dl8s5"]
Sep 18 19:13:39.054930 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:39.054895    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099537b0-bc73-4c11-8905-6424c25e40b0" path="/var/lib/kubelet/pods/099537b0-bc73-4c11-8905-6424c25e40b0/volumes"
Sep 18 19:13:39.055325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:13:39.055239    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="553fe217-88b7-41fb-bb78-c0e6371d9b78" path="/var/lib/kubelet/pods/553fe217-88b7-41fb-bb78-c0e6371d9b78/volumes"
Sep 18 19:14:31.120123 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:14:31.120093    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:14:31.125377 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:14:31.125355    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:15:00.181858 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.181826    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"]
Sep 18 19:15:00.182282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182219    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy"
Sep 18 19:15:00.182282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182232    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy"
Sep 18 19:15:00.182282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182244    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553fe217-88b7-41fb-bb78-c0e6371d9b78" containerName="revoke-keys"
Sep 18 19:15:00.182282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182250    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="553fe217-88b7-41fb-bb78-c0e6371d9b78" containerName="revoke-keys"
Sep 18 19:15:00.182282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182281    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182287    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182293    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099537b0-bc73-4c11-8905-6424c25e40b0" containerName="payload-processing"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182299    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="099537b0-bc73-4c11-8905-6424c25e40b0" containerName="payload-processing"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182373    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="099537b0-bc73-4c11-8905-6424c25e40b0" containerName="payload-processing"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182382    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182389    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182395    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="553fe217-88b7-41fb-bb78-c0e6371d9b78" containerName="revoke-keys"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182400    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy"
Sep 18 19:15:00.182448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.182407    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy"
Sep 18 19:15:00.185401 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.185382    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"
Sep 18 19:15:00.199171 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.193192    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:15:00.201692 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.201673    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"
Sep 18 19:15:00.217404 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.217381    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"]
Sep 18 19:15:00.217900 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.217881    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy"
Sep 18 19:15:00.218002 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.217902    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6fe400-5f1d-4c17-896d-e1332effb9c4" containerName="istio-proxy"
Sep 18 19:15:00.218002 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.217939    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy"
Sep 18 19:15:00.218002 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.217948    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e88335c-687d-4816-b5ca-1a42e08247d0" containerName="istio-proxy"
Sep 18 19:15:00.222029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.222008    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"
Sep 18 19:15:00.222595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.222573    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"]
Sep 18 19:15:00.243808 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.243781    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"
Sep 18 19:15:00.245286 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.245261    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"]
Sep 18 19:15:00.491119 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.491089    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"]
Sep 18 19:15:00.494306 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:15:00.494276    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72fa953_ef41_4604_b92f_dcc4510a742c.slice/crio-3b77d58bd66160d85264affb2ee3410c6b8f7c2e830e28f5dc6b8130ac6cd233 WatchSource:0}: Error finding container 3b77d58bd66160d85264affb2ee3410c6b8f7c2e830e28f5dc6b8130ac6cd233: Status 404 returned error can't find the container with id 3b77d58bd66160d85264affb2ee3410c6b8f7c2e830e28f5dc6b8130ac6cd233
Sep 18 19:15:00.542704 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:00.542683    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"]
Sep 18 19:15:00.544177 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:15:00.544156    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6ff38c_3d77_47f2_9eb5_19bfd2e2e578.slice/crio-a420e08e9f64178828e9dea61f7683050e235d4aed859c252c18ffaf36be5ea6 WatchSource:0}: Error finding container a420e08e9f64178828e9dea61f7683050e235d4aed859c252c18ffaf36be5ea6: Status 404 returned error can't find the container with id a420e08e9f64178828e9dea61f7683050e235d4aed859c252c18ffaf36be5ea6
Sep 18 19:15:01.065578 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:01.065461    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn" event={"ID":"b72fa953-ef41-4604-b92f-dcc4510a742c","Type":"ContainerStarted","Data":"8e2ecc41792f56b2d7d6cb0ec488ede6b4d87da4360f907d245965b49c4978c7"}
Sep 18 19:15:01.065578 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:01.065543    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn" event={"ID":"b72fa953-ef41-4604-b92f-dcc4510a742c","Type":"ContainerStarted","Data":"3b77d58bd66160d85264affb2ee3410c6b8f7c2e830e28f5dc6b8130ac6cd233"}
Sep 18 19:15:01.067122 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:01.067089    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2" event={"ID":"4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578","Type":"ContainerStarted","Data":"fcb736a3db1b1217f7fa11add44df66c094a9522c70a4842ceaac3c168da646f"}
Sep 18 19:15:01.067246 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:01.067132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2" event={"ID":"4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578","Type":"ContainerStarted","Data":"a420e08e9f64178828e9dea61f7683050e235d4aed859c252c18ffaf36be5ea6"}
Sep 18 19:15:01.083331 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:01.083286    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn" podStartSLOduration=1.083274602 podStartE2EDuration="1.083274602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:01.081751338 +0000 UTC m=+1830.520011542" watchObservedRunningTime="2026-09-18 19:15:01.083274602 +0000 UTC m=+1830.521534806"
Sep 18 19:15:01.097012 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:01.096973    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2" podStartSLOduration=1.096959922 podStartE2EDuration="1.096959922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:15:01.096437236 +0000 UTC m=+1830.534697445" watchObservedRunningTime="2026-09-18 19:15:01.096959922 +0000 UTC m=+1830.535220126"
Sep 18 19:15:02.073425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:02.073392    2568 generic.go:356] "Generic (PLEG): container finished" podID="4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578" containerID="fcb736a3db1b1217f7fa11add44df66c094a9522c70a4842ceaac3c168da646f" exitCode=0
Sep 18 19:15:02.073917 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:02.073482    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2" event={"ID":"4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578","Type":"ContainerDied","Data":"fcb736a3db1b1217f7fa11add44df66c094a9522c70a4842ceaac3c168da646f"}
Sep 18 19:15:02.075023 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:02.075003    2568 generic.go:356] "Generic (PLEG): container finished" podID="b72fa953-ef41-4604-b92f-dcc4510a742c" containerID="8e2ecc41792f56b2d7d6cb0ec488ede6b4d87da4360f907d245965b49c4978c7" exitCode=0
Sep 18 19:15:02.075124 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:02.075048    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn" event={"ID":"b72fa953-ef41-4604-b92f-dcc4510a742c","Type":"ContainerDied","Data":"8e2ecc41792f56b2d7d6cb0ec488ede6b4d87da4360f907d245965b49c4978c7"}
Sep 18 19:15:03.234433 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:03.234404    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"
Sep 18 19:15:03.237195 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:03.237176    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"
Sep 18 19:15:04.086841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:04.086810    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2"
Sep 18 19:15:04.087003 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:04.086844    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829315-wvbh2" event={"ID":"4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578","Type":"ContainerDied","Data":"a420e08e9f64178828e9dea61f7683050e235d4aed859c252c18ffaf36be5ea6"}
Sep 18 19:15:04.087003 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:04.086894    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a420e08e9f64178828e9dea61f7683050e235d4aed859c252c18ffaf36be5ea6"
Sep 18 19:15:04.088099 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:04.088079    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"
Sep 18 19:15:04.088220 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:04.088108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn" event={"ID":"b72fa953-ef41-4604-b92f-dcc4510a742c","Type":"ContainerDied","Data":"3b77d58bd66160d85264affb2ee3410c6b8f7c2e830e28f5dc6b8130ac6cd233"}
Sep 18 19:15:04.088220 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:04.088145    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b77d58bd66160d85264affb2ee3410c6b8f7c2e830e28f5dc6b8130ac6cd233"
Sep 18 19:15:31.471066 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:15:31.471034    2568 scope.go:117] "RemoveContainer" containerID="9e6f2724ce8dc098c3bce6fc55cf094f1fe0e6f3a6cec8fc4707eccf878e1ae5"
Sep 18 19:16:41.254032 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.253997    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"]
Sep 18 19:16:41.256378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.254408    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578" containerName="cleanup"
Sep 18 19:16:41.256378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.254419    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578" containerName="cleanup"
Sep 18 19:16:41.256378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.254440    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72fa953-ef41-4604-b92f-dcc4510a742c" containerName="cleanup"
Sep 18 19:16:41.256378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.254445    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72fa953-ef41-4604-b92f-dcc4510a742c" containerName="cleanup"
Sep 18 19:16:41.256378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.254510    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578" containerName="cleanup"
Sep 18 19:16:41.256378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.254538    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72fa953-ef41-4604-b92f-dcc4510a742c" containerName="cleanup"
Sep 18 19:16:41.257173 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.257157    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.260062 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.260040    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f8cdfc\""
Sep 18 19:16:41.289829 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.289808    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"]
Sep 18 19:16:41.299029 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.299001    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d18a88-a4e1-4289-98e8-c0fc840894cc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.299164 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.299050    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6lhl\" (UniqueName: \"kubernetes.io/projected/80d18a88-a4e1-4289-98e8-c0fc840894cc-kube-api-access-p6lhl\") pod \"payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.400233 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.400200    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d18a88-a4e1-4289-98e8-c0fc840894cc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.400383 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.400246    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6lhl\" (UniqueName: \"kubernetes.io/projected/80d18a88-a4e1-4289-98e8-c0fc840894cc-kube-api-access-p6lhl\") pod \"payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.400970 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.400951    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d18a88-a4e1-4289-98e8-c0fc840894cc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.409807 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.409776    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6lhl\" (UniqueName: \"kubernetes.io/projected/80d18a88-a4e1-4289-98e8-c0fc840894cc-kube-api-access-p6lhl\") pod \"payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.567959 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.567875    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:41.934743 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:41.934663    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"]
Sep 18 19:16:41.968911 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:16:41.968883    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d18a88_a4e1_4289_98e8_c0fc840894cc.slice/crio-837cb493bfaf75d93ba5f777b703057b91bac7dc8d9695b22bc22bd550a8f124 WatchSource:0}: Error finding container 837cb493bfaf75d93ba5f777b703057b91bac7dc8d9695b22bc22bd550a8f124: Status 404 returned error can't find the container with id 837cb493bfaf75d93ba5f777b703057b91bac7dc8d9695b22bc22bd550a8f124
Sep 18 19:16:42.628938 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:42.628904    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w" event={"ID":"80d18a88-a4e1-4289-98e8-c0fc840894cc","Type":"ContainerStarted","Data":"cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf"}
Sep 18 19:16:42.629325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:42.628958    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w" event={"ID":"80d18a88-a4e1-4289-98e8-c0fc840894cc","Type":"ContainerStarted","Data":"837cb493bfaf75d93ba5f777b703057b91bac7dc8d9695b22bc22bd550a8f124"}
Sep 18 19:16:42.629325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:42.629006    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:16:42.656828 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:42.656779    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w" podStartSLOduration=1.656766446 podStartE2EDuration="1.656766446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:16:42.654320332 +0000 UTC m=+1932.092580536" watchObservedRunningTime="2026-09-18 19:16:42.656766446 +0000 UTC m=+1932.095026649"
Sep 18 19:16:53.636862 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:16:53.636826    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:18:29.173662 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:29.173609    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w_80d18a88-a4e1-4289-98e8-c0fc840894cc/payload-processing/0.log"
Sep 18 19:18:30.382592 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.382537    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"]
Sep 18 19:18:30.386604 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.386583    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.392727 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.392707    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-790775-openshift-default-dockercfg-f67j8\""
Sep 18 19:18:30.409497 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.409474    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"]
Sep 18 19:18:30.536564 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536524    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-data\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536739 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-credential-socket\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536739 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-socket\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536739 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536645    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-envoy\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536739 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536699    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-podinfo\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536739 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536715    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvk4\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-kube-api-access-djvk4\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536906 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536748    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-certs\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536906 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-token\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.536906 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.536776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-data\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638218    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-credential-socket\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638234    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-socket\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638570 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638272    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-envoy\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638570 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638308    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-podinfo\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638570 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638325    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvk4\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-kube-api-access-djvk4\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638570 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638349    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-certs\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638570 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-token\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638570 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638884 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638687    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-data\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638884 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638721    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-socket\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.638884 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.638748    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-credential-socket\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.639039 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.639010    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-certs\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.639100 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.639075    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.640729 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.640709    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-envoy\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.640856 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.640839    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-podinfo\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.650375 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.650344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-token\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.650554 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.650537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvk4\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-kube-api-access-djvk4\") pod \"e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") " pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:30.697569 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:30.697542    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:31.045210 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:18:31.045181    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b2587b_2c04_4c53_a7c5_aac31f50c0b0.slice/crio-af9ad9bf03c5ddf393e822cfeac47ba52fa8be7fe4e3ce5b5dd5739002c3d09d WatchSource:0}: Error finding container af9ad9bf03c5ddf393e822cfeac47ba52fa8be7fe4e3ce5b5dd5739002c3d09d: Status 404 returned error can't find the container with id af9ad9bf03c5ddf393e822cfeac47ba52fa8be7fe4e3ce5b5dd5739002c3d09d
Sep 18 19:18:31.047466 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.047421    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:18:31.047559 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.047484    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:18:31.047559 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.047491    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"]
Sep 18 19:18:31.047559 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.047535    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:18:31.206230 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.206193    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" event={"ID":"65b2587b-2c04-4c53-a7c5-aac31f50c0b0","Type":"ContainerStarted","Data":"1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"}
Sep 18 19:18:31.206426 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.206251    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" event={"ID":"65b2587b-2c04-4c53-a7c5-aac31f50c0b0","Type":"ContainerStarted","Data":"af9ad9bf03c5ddf393e822cfeac47ba52fa8be7fe4e3ce5b5dd5739002c3d09d"}
Sep 18 19:18:31.277793 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.277724    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podStartSLOduration=1.277707189 podStartE2EDuration="1.277707189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:18:31.277656658 +0000 UTC m=+2040.715916863" watchObservedRunningTime="2026-09-18 19:18:31.277707189 +0000 UTC m=+2040.715967449"
Sep 18 19:18:31.515086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.515056    2568 scope.go:117] "RemoveContainer" containerID="c11ac4449f85b76bfcbfc26bd2a09170414615b164845a8e7774d2dd9c9c9043"
Sep 18 19:18:31.525173 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.525148    2568 scope.go:117] "RemoveContainer" containerID="46d4874745a34f51493e389dc79018b41948b4a1e8e835e1a62e30d5a8f9c00f"
Sep 18 19:18:31.698006 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:31.697978    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:32.703330 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:32.703302    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:33.218109 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:33.218080    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:33.218932 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:33.218913    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:41.953485 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:41.953450    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"]
Sep 18 19:18:41.957456 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:41.957435    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:41.960128 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:41.960107    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-790775\""
Sep 18 19:18:41.960302 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:41.960285    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-790775-dockercfg-cq6wz\""
Sep 18 19:18:41.971220 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:41.971196    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"]
Sep 18 19:18:42.054151 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.054117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7706097c-351a-4493-a72e-4d8f214410e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.054305 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.054202    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h7ph\" (UniqueName: \"kubernetes.io/projected/7706097c-351a-4493-a72e-4d8f214410e7-kube-api-access-9h7ph\") pod \"payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.155636 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.155606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7706097c-351a-4493-a72e-4d8f214410e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.155798 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.155667    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h7ph\" (UniqueName: \"kubernetes.io/projected/7706097c-351a-4493-a72e-4d8f214410e7-kube-api-access-9h7ph\") pod \"payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.156264 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.156243    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7706097c-351a-4493-a72e-4d8f214410e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.168235 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.168208    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h7ph\" (UniqueName: \"kubernetes.io/projected/7706097c-351a-4493-a72e-4d8f214410e7-kube-api-access-9h7ph\") pod \"payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.268525 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.268494    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:42.599702 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:42.599647    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"]
Sep 18 19:18:42.602155 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:18:42.602129    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7706097c_351a_4493_a72e_4d8f214410e7.slice/crio-a270d48010317ede6cf96149ce96d30f0893d68a7b9a2b599344e91fc7e9301f WatchSource:0}: Error finding container a270d48010317ede6cf96149ce96d30f0893d68a7b9a2b599344e91fc7e9301f: Status 404 returned error can't find the container with id a270d48010317ede6cf96149ce96d30f0893d68a7b9a2b599344e91fc7e9301f
Sep 18 19:18:43.273839 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:43.273804    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4" event={"ID":"7706097c-351a-4493-a72e-4d8f214410e7","Type":"ContainerStarted","Data":"0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428"}
Sep 18 19:18:43.274225 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:43.273864    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4" event={"ID":"7706097c-351a-4493-a72e-4d8f214410e7","Type":"ContainerStarted","Data":"a270d48010317ede6cf96149ce96d30f0893d68a7b9a2b599344e91fc7e9301f"}
Sep 18 19:18:43.274225 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:43.273887    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:54.282160 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:54.282085    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:18:54.317842 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:54.317797    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4" podStartSLOduration=13.317784536 podStartE2EDuration="13.317784536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:18:43.346076583 +0000 UTC m=+2052.784336789" watchObservedRunningTime="2026-09-18 19:18:54.317784536 +0000 UTC m=+2063.756044740"
Sep 18 19:18:56.351418 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:56.351385    2568 generic.go:356] "Generic (PLEG): container finished" podID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerID="1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9" exitCode=0
Sep 18 19:18:56.351847 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:56.351463    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" event={"ID":"65b2587b-2c04-4c53-a7c5-aac31f50c0b0","Type":"ContainerDied","Data":"1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"}
Sep 18 19:18:56.351922 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:56.351907    2568 scope.go:117] "RemoveContainer" containerID="1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"
Sep 18 19:18:56.352985 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:56.352959    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:18:56.353044 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:56.353022    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:18:56.353080 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:56.353054    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 19:18:57.358838 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:57.358802    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" event={"ID":"65b2587b-2c04-4c53-a7c5-aac31f50c0b0","Type":"ContainerStarted","Data":"9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728"}
Sep 18 19:18:57.698814 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:57.698719    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:18:57.698996 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:57.698958    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 18 19:18:57.699077 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:57.699025    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 18 19:18:58.698505 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:58.698464    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 18 19:18:58.698886 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:58.698557    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 18 19:18:59.698316 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:59.698282    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 18 19:18:59.698485 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:18:59.698336    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 18 19:19:00.698181 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:00.698147    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 18 19:19:00.698603 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:00.698203    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 18 19:19:01.698915 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:01.698874    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 18 19:19:01.699382 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:01.698936    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 18 19:19:02.701890 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:02.701863    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:19:02.702323 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:02.702302    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:19:02.702900 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:02.702878    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:19:09.473469 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.473433    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"]
Sep 18 19:19:09.476111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.476092    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:09.483847 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.483819    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:19:09.484541 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.484505    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:19:09.522473 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.522441    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"]
Sep 18 19:19:09.649902 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.649854    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae31318-c212-40b2-bcc8-b3754899a59d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2\" (UID: \"6ae31318-c212-40b2-bcc8-b3754899a59d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:09.751300 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.751201    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae31318-c212-40b2-bcc8-b3754899a59d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2\" (UID: \"6ae31318-c212-40b2-bcc8-b3754899a59d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:09.751891 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.751872    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae31318-c212-40b2-bcc8-b3754899a59d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2\" (UID: \"6ae31318-c212-40b2-bcc8-b3754899a59d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:09.787255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:09.787205    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:10.123758 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:10.123730    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"]
Sep 18 19:19:10.125508 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:19:10.125483    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae31318_c212_40b2_bcc8_b3754899a59d.slice/crio-e6959abe7095e882f054f890b1d3400b9580cd7f78052050d0860b1d3d39b59f WatchSource:0}: Error finding container e6959abe7095e882f054f890b1d3400b9580cd7f78052050d0860b1d3d39b59f: Status 404 returned error can't find the container with id e6959abe7095e882f054f890b1d3400b9580cd7f78052050d0860b1d3d39b59f
Sep 18 19:19:10.432064 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:10.431979    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2" event={"ID":"6ae31318-c212-40b2-bcc8-b3754899a59d","Type":"ContainerStarted","Data":"aeb55b5f9fc5fa2e7b68609bd6cb5e9f7c6cc010d65b67844d5790eefc37813b"}
Sep 18 19:19:10.432064 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:10.432044    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2" event={"ID":"6ae31318-c212-40b2-bcc8-b3754899a59d","Type":"ContainerStarted","Data":"e6959abe7095e882f054f890b1d3400b9580cd7f78052050d0860b1d3d39b59f"}
Sep 18 19:19:10.462558 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:10.462494    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2" podStartSLOduration=1.462479376 podStartE2EDuration="1.462479376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:19:10.462405118 +0000 UTC m=+2079.900665324" watchObservedRunningTime="2026-09-18 19:19:10.462479376 +0000 UTC m=+2079.900739581"
Sep 18 19:19:11.440248 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:11.440217    2568 generic.go:356] "Generic (PLEG): container finished" podID="6ae31318-c212-40b2-bcc8-b3754899a59d" containerID="aeb55b5f9fc5fa2e7b68609bd6cb5e9f7c6cc010d65b67844d5790eefc37813b" exitCode=0
Sep 18 19:19:11.440659 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:11.440265    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2" event={"ID":"6ae31318-c212-40b2-bcc8-b3754899a59d","Type":"ContainerDied","Data":"aeb55b5f9fc5fa2e7b68609bd6cb5e9f7c6cc010d65b67844d5790eefc37813b"}
Sep 18 19:19:12.585623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:12.585600    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:12.680835 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:12.680799    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae31318-c212-40b2-bcc8-b3754899a59d-maas-api-service-ca\") pod \"6ae31318-c212-40b2-bcc8-b3754899a59d\" (UID: \"6ae31318-c212-40b2-bcc8-b3754899a59d\") "
Sep 18 19:19:12.681129 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:12.681106    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae31318-c212-40b2-bcc8-b3754899a59d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ae31318-c212-40b2-bcc8-b3754899a59d" (UID: "6ae31318-c212-40b2-bcc8-b3754899a59d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:19:12.681240 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:12.681218    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae31318-c212-40b2-bcc8-b3754899a59d-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:13.452889 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:13.452796    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2" event={"ID":"6ae31318-c212-40b2-bcc8-b3754899a59d","Type":"ContainerDied","Data":"e6959abe7095e882f054f890b1d3400b9580cd7f78052050d0860b1d3d39b59f"}
Sep 18 19:19:13.452889 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:13.452846    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6959abe7095e882f054f890b1d3400b9580cd7f78052050d0860b1d3d39b59f"
Sep 18 19:19:13.452889 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:13.452805    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"
Sep 18 19:19:21.019442 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.019408    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"]
Sep 18 19:19:21.019937 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.019657    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4" podUID="7706097c-351a-4493-a72e-4d8f214410e7" containerName="payload-pre-processing" containerID="cri-o://0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428" gracePeriod=30
Sep 18 19:19:21.281484 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.281419    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:19:21.374260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.374224    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h7ph\" (UniqueName: \"kubernetes.io/projected/7706097c-351a-4493-a72e-4d8f214410e7-kube-api-access-9h7ph\") pod \"7706097c-351a-4493-a72e-4d8f214410e7\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") "
Sep 18 19:19:21.374427 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.374298    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7706097c-351a-4493-a72e-4d8f214410e7-plugins-config-volume\") pod \"7706097c-351a-4493-a72e-4d8f214410e7\" (UID: \"7706097c-351a-4493-a72e-4d8f214410e7\") "
Sep 18 19:19:21.374682 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.374658    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7706097c-351a-4493-a72e-4d8f214410e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7706097c-351a-4493-a72e-4d8f214410e7" (UID: "7706097c-351a-4493-a72e-4d8f214410e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:19:21.376402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.376375    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7706097c-351a-4493-a72e-4d8f214410e7-kube-api-access-9h7ph" (OuterVolumeSpecName: "kube-api-access-9h7ph") pod "7706097c-351a-4493-a72e-4d8f214410e7" (UID: "7706097c-351a-4493-a72e-4d8f214410e7"). InnerVolumeSpecName "kube-api-access-9h7ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:19:21.476370 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.476283    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7706097c-351a-4493-a72e-4d8f214410e7-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:21.476370 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.476324    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h7ph\" (UniqueName: \"kubernetes.io/projected/7706097c-351a-4493-a72e-4d8f214410e7-kube-api-access-9h7ph\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:21.498800 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.498764    2568 generic.go:356] "Generic (PLEG): container finished" podID="7706097c-351a-4493-a72e-4d8f214410e7" containerID="0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428" exitCode=0
Sep 18 19:19:21.498975 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.498811    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"
Sep 18 19:19:21.498975 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.498849    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4" event={"ID":"7706097c-351a-4493-a72e-4d8f214410e7","Type":"ContainerDied","Data":"0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428"}
Sep 18 19:19:21.498975 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.498908    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4" event={"ID":"7706097c-351a-4493-a72e-4d8f214410e7","Type":"ContainerDied","Data":"a270d48010317ede6cf96149ce96d30f0893d68a7b9a2b599344e91fc7e9301f"}
Sep 18 19:19:21.498975 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.498931    2568 scope.go:117] "RemoveContainer" containerID="0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428"
Sep 18 19:19:21.509207 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.509186    2568 scope.go:117] "RemoveContainer" containerID="0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428"
Sep 18 19:19:21.509553 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:19:21.509528    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428\": container with ID starting with 0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428 not found: ID does not exist" containerID="0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428"
Sep 18 19:19:21.509632 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.509566    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428"} err="failed to get container status \"0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428\": rpc error: code = NotFound desc = could not find container \"0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428\": container with ID starting with 0102d5d11fdac37c708519c25ffbce43f49e4478e2f5272ea71a633d982b0428 not found: ID does not exist"
Sep 18 19:19:21.543919 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.543827    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"]
Sep 18 19:19:21.549728 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.549696    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-790775-xmvf2"]
Sep 18 19:19:21.563876 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.563842    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"]
Sep 18 19:19:21.597379 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:21.597346    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-790775-7dcf494dc4-dk2m4"]
Sep 18 19:19:23.055645 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:23.055615    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae31318-c212-40b2-bcc8-b3754899a59d" path="/var/lib/kubelet/pods/6ae31318-c212-40b2-bcc8-b3754899a59d/volumes"
Sep 18 19:19:23.056065 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:23.056045    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7706097c-351a-4493-a72e-4d8f214410e7" path="/var/lib/kubelet/pods/7706097c-351a-4493-a72e-4d8f214410e7/volumes"
Sep 18 19:19:31.157931 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:31.157903    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:19:31.163130 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:31.163105    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:19:31.550562 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:31.550530    2568 scope.go:117] "RemoveContainer" containerID="9bbb9302f05d904ec07207fe476c08db62d4f629e9e84c735d3fa2e693e9bbd3"
Sep 18 19:19:31.559777 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:31.559751    2568 scope.go:117] "RemoveContainer" containerID="c24acc6fc672373f56292ea2c76ac4f0830f9b4031bb489461d6b09a7cb54ffd"
Sep 18 19:19:31.567667 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:31.567649    2568 scope.go:117] "RemoveContainer" containerID="8348b0324c5df765998a03679b13fa2eb15c8a4c240ca47559452d2981e1280e"
Sep 18 19:19:31.578760 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:31.578742    2568 scope.go:117] "RemoveContainer" containerID="a3f047950f7b318768494a30a8bcf47917ac1ebc9fb34616dbd5a6d2f409444d"
Sep 18 19:19:32.162331 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:32.162295    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"]
Sep 18 19:19:32.162815 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:32.162561    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" containerID="cri-o://9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728" gracePeriod=30
Sep 18 19:19:32.705165 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:32.705127    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 19:19:32.705347 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:32.705187    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:19:36.185609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.185573    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"]
Sep 18 19:19:36.186019 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.186007    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7706097c-351a-4493-a72e-4d8f214410e7" containerName="payload-pre-processing"
Sep 18 19:19:36.186066 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.186021    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7706097c-351a-4493-a72e-4d8f214410e7" containerName="payload-pre-processing"
Sep 18 19:19:36.186066 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.186036    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae31318-c212-40b2-bcc8-b3754899a59d" containerName="revoke-keys"
Sep 18 19:19:36.186066 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.186042    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae31318-c212-40b2-bcc8-b3754899a59d" containerName="revoke-keys"
Sep 18 19:19:36.186160 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.186107    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7706097c-351a-4493-a72e-4d8f214410e7" containerName="payload-pre-processing"
Sep 18 19:19:36.186160 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.186117    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae31318-c212-40b2-bcc8-b3754899a59d" containerName="revoke-keys"
Sep 18 19:19:36.188480 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.188462    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:36.192041 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.192007    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:19:36.193265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.193249    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:19:36.207575 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.207532    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"]
Sep 18 19:19:36.327012 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.326963    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc4656d-421f-4833-af78-d4ee4ac3f404-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg\" (UID: \"afc4656d-421f-4833-af78-d4ee4ac3f404\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:36.428581 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.428547    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc4656d-421f-4833-af78-d4ee4ac3f404-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg\" (UID: \"afc4656d-421f-4833-af78-d4ee4ac3f404\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:36.429252 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.429227    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc4656d-421f-4833-af78-d4ee4ac3f404-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg\" (UID: \"afc4656d-421f-4833-af78-d4ee4ac3f404\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:36.498493 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.498466    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:36.785373 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:36.785349    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"]
Sep 18 19:19:36.788259 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:19:36.788234    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc4656d_421f_4833_af78_d4ee4ac3f404.slice/crio-42d07c90195d7a214d3e9a8c73a663a75c4b3fb3f03ad699fad345f687909b3f WatchSource:0}: Error finding container 42d07c90195d7a214d3e9a8c73a663a75c4b3fb3f03ad699fad345f687909b3f: Status 404 returned error can't find the container with id 42d07c90195d7a214d3e9a8c73a663a75c4b3fb3f03ad699fad345f687909b3f
Sep 18 19:19:37.423662 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.423635    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:19:37.540224 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540200    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istiod-ca-cert\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540233    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-podinfo\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540325 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540252    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-token\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540415 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540331    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-data\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540415 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540357    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djvk4\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-kube-api-access-djvk4\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540415 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540395    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-certs\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540461    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-credential-socket\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540490    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-socket\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540607    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-envoy\") pod \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\" (UID: \"65b2587b-2c04-4c53-a7c5-aac31f50c0b0\") "
Sep 18 19:19:37.540782 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540631    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-data" (OuterVolumeSpecName: "istio-data") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:19:37.540782 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540678    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:19:37.540782 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540679    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:19:37.540937 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540844    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:19:37.540937 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.540881    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:19:37.541085 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.541067    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.541137 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.541093    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.541137 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.541108    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.541137 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.541124    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.541231 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.541138    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.542378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.542336    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:19:37.542478 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.542411    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-kube-api-access-djvk4" (OuterVolumeSpecName: "kube-api-access-djvk4") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "kube-api-access-djvk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:19:37.542738 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.542719    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:19:37.542845 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.542832    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65b2587b-2c04-4c53-a7c5-aac31f50c0b0" (UID: "65b2587b-2c04-4c53-a7c5-aac31f50c0b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:19:37.589920 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.589890    2568 generic.go:356] "Generic (PLEG): container finished" podID="afc4656d-421f-4833-af78-d4ee4ac3f404" containerID="b7cfe3e9859c866a9b3b33bc072dc0b8d2d0d90e84ffe415605b3c2f08a2fe0f" exitCode=0
Sep 18 19:19:37.590061 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.589971    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg" event={"ID":"afc4656d-421f-4833-af78-d4ee4ac3f404","Type":"ContainerDied","Data":"b7cfe3e9859c866a9b3b33bc072dc0b8d2d0d90e84ffe415605b3c2f08a2fe0f"}
Sep 18 19:19:37.590135 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.590037    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg" event={"ID":"afc4656d-421f-4833-af78-d4ee4ac3f404","Type":"ContainerStarted","Data":"42d07c90195d7a214d3e9a8c73a663a75c4b3fb3f03ad699fad345f687909b3f"}
Sep 18 19:19:37.591876 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.591847    2568 generic.go:356] "Generic (PLEG): container finished" podID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerID="9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728" exitCode=0
Sep 18 19:19:37.591991 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.591886    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"
Sep 18 19:19:37.591991 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.591936    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" event={"ID":"65b2587b-2c04-4c53-a7c5-aac31f50c0b0","Type":"ContainerDied","Data":"9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728"}
Sep 18 19:19:37.591991 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.591981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49" event={"ID":"65b2587b-2c04-4c53-a7c5-aac31f50c0b0","Type":"ContainerDied","Data":"af9ad9bf03c5ddf393e822cfeac47ba52fa8be7fe4e3ce5b5dd5739002c3d09d"}
Sep 18 19:19:37.592098 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.592005    2568 scope.go:117] "RemoveContainer" containerID="9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728"
Sep 18 19:19:37.601319 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.601302    2568 scope.go:117] "RemoveContainer" containerID="1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"
Sep 18 19:19:37.609652 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.609636    2568 scope.go:117] "RemoveContainer" containerID="9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728"
Sep 18 19:19:37.609903 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:19:37.609887    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728\": container with ID starting with 9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728 not found: ID does not exist" containerID="9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728"
Sep 18 19:19:37.609946 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.609913    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728"} err="failed to get container status \"9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728\": rpc error: code = NotFound desc = could not find container \"9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728\": container with ID starting with 9e9a36cf1acbc0c2a40cb4c1e07d68598fa7432f574fadc6583bcd1e26310728 not found: ID does not exist"
Sep 18 19:19:37.609946 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.609929    2568 scope.go:117] "RemoveContainer" containerID="1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"
Sep 18 19:19:37.610173 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:19:37.610156    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9\": container with ID starting with 1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9 not found: ID does not exist" containerID="1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"
Sep 18 19:19:37.610209 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.610178    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9"} err="failed to get container status \"1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9\": rpc error: code = NotFound desc = could not find container \"1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9\": container with ID starting with 1c314177b74082d8ad8dbac815242ec5afddd0a2a157cef5ef9e6187f032f4f9 not found: ID does not exist"
Sep 18 19:19:37.641969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.641942    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djvk4\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-kube-api-access-djvk4\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.641969 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.641968    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.642150 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.641978    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.642150 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.641985    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b2587b-2c04-4c53-a7c5-aac31f50c0b0-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:37.665048 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.665023    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"]
Sep 18 19:19:37.668903 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:37.668881    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-790775-openshift-default-86bfc84fbb-s4s49"]
Sep 18 19:19:38.749697 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:38.749669    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:38.853462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:38.853374    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc4656d-421f-4833-af78-d4ee4ac3f404-maas-api-service-ca\") pod \"afc4656d-421f-4833-af78-d4ee4ac3f404\" (UID: \"afc4656d-421f-4833-af78-d4ee4ac3f404\") "
Sep 18 19:19:38.853736 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:38.853715    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc4656d-421f-4833-af78-d4ee4ac3f404-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "afc4656d-421f-4833-af78-d4ee4ac3f404" (UID: "afc4656d-421f-4833-af78-d4ee4ac3f404"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:19:38.955014 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:38.954977    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc4656d-421f-4833-af78-d4ee4ac3f404-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:39.055552 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:39.055503    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" path="/var/lib/kubelet/pods/65b2587b-2c04-4c53-a7c5-aac31f50c0b0/volumes"
Sep 18 19:19:39.605388 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:39.605353    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"
Sep 18 19:19:39.605567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:39.605385    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg" event={"ID":"afc4656d-421f-4833-af78-d4ee4ac3f404","Type":"ContainerDied","Data":"42d07c90195d7a214d3e9a8c73a663a75c4b3fb3f03ad699fad345f687909b3f"}
Sep 18 19:19:39.605567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:39.605432    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42d07c90195d7a214d3e9a8c73a663a75c4b3fb3f03ad699fad345f687909b3f"
Sep 18 19:19:52.298742 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.298709    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"]
Sep 18 19:19:52.299132 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.298899    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w" podUID="80d18a88-a4e1-4289-98e8-c0fc840894cc" containerName="payload-processing" containerID="cri-o://cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf" gracePeriod=30
Sep 18 19:19:52.544922 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.544891    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:19:52.675411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.675324    2568 generic.go:356] "Generic (PLEG): container finished" podID="80d18a88-a4e1-4289-98e8-c0fc840894cc" containerID="cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf" exitCode=0
Sep 18 19:19:52.675411 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.675392    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"
Sep 18 19:19:52.675640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.675416    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w" event={"ID":"80d18a88-a4e1-4289-98e8-c0fc840894cc","Type":"ContainerDied","Data":"cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf"}
Sep 18 19:19:52.675640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.675487    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w" event={"ID":"80d18a88-a4e1-4289-98e8-c0fc840894cc","Type":"ContainerDied","Data":"837cb493bfaf75d93ba5f777b703057b91bac7dc8d9695b22bc22bd550a8f124"}
Sep 18 19:19:52.675640 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.675549    2568 scope.go:117] "RemoveContainer" containerID="cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf"
Sep 18 19:19:52.686727 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.686698    2568 scope.go:117] "RemoveContainer" containerID="cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf"
Sep 18 19:19:52.689936 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:19:52.689899    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf\": container with ID starting with cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf not found: ID does not exist" containerID="cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf"
Sep 18 19:19:52.690072 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.689950    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf"} err="failed to get container status \"cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf\": rpc error: code = NotFound desc = could not find container \"cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf\": container with ID starting with cf51964c84daf28e9146bd2b219558f48c00feca1b72330e7bd7ae2c6cdbe5cf not found: ID does not exist"
Sep 18 19:19:52.691307 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.691285    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d18a88-a4e1-4289-98e8-c0fc840894cc-plugins-config-volume\") pod \"80d18a88-a4e1-4289-98e8-c0fc840894cc\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") "
Sep 18 19:19:52.691417 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.691356    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6lhl\" (UniqueName: \"kubernetes.io/projected/80d18a88-a4e1-4289-98e8-c0fc840894cc-kube-api-access-p6lhl\") pod \"80d18a88-a4e1-4289-98e8-c0fc840894cc\" (UID: \"80d18a88-a4e1-4289-98e8-c0fc840894cc\") "
Sep 18 19:19:52.691771 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.691748    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d18a88-a4e1-4289-98e8-c0fc840894cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80d18a88-a4e1-4289-98e8-c0fc840894cc" (UID: "80d18a88-a4e1-4289-98e8-c0fc840894cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:19:52.694027 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.693990    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d18a88-a4e1-4289-98e8-c0fc840894cc-kube-api-access-p6lhl" (OuterVolumeSpecName: "kube-api-access-p6lhl") pod "80d18a88-a4e1-4289-98e8-c0fc840894cc" (UID: "80d18a88-a4e1-4289-98e8-c0fc840894cc"). InnerVolumeSpecName "kube-api-access-p6lhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:19:52.792894 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.792856    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d18a88-a4e1-4289-98e8-c0fc840894cc-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:52.792894 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.792886    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6lhl\" (UniqueName: \"kubernetes.io/projected/80d18a88-a4e1-4289-98e8-c0fc840894cc-kube-api-access-p6lhl\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:19:52.867577 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.867540    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"]
Sep 18 19:19:52.873880 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:52.873849    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f8cdfc-wpfwg"]
Sep 18 19:19:53.035379 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:53.035345    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"]
Sep 18 19:19:53.055417 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:53.055372    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afc4656d-421f-4833-af78-d4ee4ac3f404" path="/var/lib/kubelet/pods/afc4656d-421f-4833-af78-d4ee4ac3f404/volumes"
Sep 18 19:19:53.085721 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:53.085674    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f8cdfc-db59846bf-l6t8w"]
Sep 18 19:19:55.054690 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:19:55.054661    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d18a88-a4e1-4289-98e8-c0fc840894cc" path="/var/lib/kubelet/pods/80d18a88-a4e1-4289-98e8-c0fc840894cc/volumes"
Sep 18 19:20:46.174765 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.174728    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"]
Sep 18 19:20:46.175211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175130    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy"
Sep 18 19:20:46.175211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175140    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy"
Sep 18 19:20:46.175211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175171    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d18a88-a4e1-4289-98e8-c0fc840894cc" containerName="payload-processing"
Sep 18 19:20:46.175211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175177    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d18a88-a4e1-4289-98e8-c0fc840894cc" containerName="payload-processing"
Sep 18 19:20:46.175211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175189    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc4656d-421f-4833-af78-d4ee4ac3f404" containerName="revoke-keys"
Sep 18 19:20:46.175211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175194    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc4656d-421f-4833-af78-d4ee4ac3f404" containerName="revoke-keys"
Sep 18 19:20:46.175461 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175259    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d18a88-a4e1-4289-98e8-c0fc840894cc" containerName="payload-processing"
Sep 18 19:20:46.175461 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175269    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="afc4656d-421f-4833-af78-d4ee4ac3f404" containerName="revoke-keys"
Sep 18 19:20:46.175461 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175278    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy"
Sep 18 19:20:46.175461 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.175284    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy"
Sep 18 19:20:46.177595 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.177578    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:46.182258 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.182237    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:20:46.183012 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.182996    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:20:46.216424 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.216394    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"]
Sep 18 19:20:46.301282 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.301248    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4243b04b-76e1-4efe-8985-4f0cbe3b3c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd\" (UID: \"4243b04b-76e1-4efe-8985-4f0cbe3b3c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:46.401992 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.401966    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4243b04b-76e1-4efe-8985-4f0cbe3b3c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd\" (UID: \"4243b04b-76e1-4efe-8985-4f0cbe3b3c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:46.402559 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.402510    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4243b04b-76e1-4efe-8985-4f0cbe3b3c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd\" (UID: \"4243b04b-76e1-4efe-8985-4f0cbe3b3c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:46.487467 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.487437    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:46.776946 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.776872    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"]
Sep 18 19:20:46.784209 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:20:46.784179    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4243b04b_76e1_4efe_8985_4f0cbe3b3c27.slice/crio-9b566a7a6d6bc46e6f570dd0d8e9e11c92bbbbe4746a2a4caa75d1afb0984947 WatchSource:0}: Error finding container 9b566a7a6d6bc46e6f570dd0d8e9e11c92bbbbe4746a2a4caa75d1afb0984947: Status 404 returned error can't find the container with id 9b566a7a6d6bc46e6f570dd0d8e9e11c92bbbbe4746a2a4caa75d1afb0984947
Sep 18 19:20:46.971232 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:46.971195    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd" event={"ID":"4243b04b-76e1-4efe-8985-4f0cbe3b3c27","Type":"ContainerStarted","Data":"9b566a7a6d6bc46e6f570dd0d8e9e11c92bbbbe4746a2a4caa75d1afb0984947"}
Sep 18 19:20:47.977625 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:47.977594    2568 generic.go:356] "Generic (PLEG): container finished" podID="4243b04b-76e1-4efe-8985-4f0cbe3b3c27" containerID="04e562f26475715275a7376f7862386648b82ad0cea6770634982e24b88d723f" exitCode=0
Sep 18 19:20:47.977996 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:47.977660    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd" event={"ID":"4243b04b-76e1-4efe-8985-4f0cbe3b3c27","Type":"ContainerDied","Data":"04e562f26475715275a7376f7862386648b82ad0cea6770634982e24b88d723f"}
Sep 18 19:20:49.117201 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.117177    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:49.229221 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.229193    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4243b04b-76e1-4efe-8985-4f0cbe3b3c27-maas-api-service-ca\") pod \"4243b04b-76e1-4efe-8985-4f0cbe3b3c27\" (UID: \"4243b04b-76e1-4efe-8985-4f0cbe3b3c27\") "
Sep 18 19:20:49.229503 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.229482    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4243b04b-76e1-4efe-8985-4f0cbe3b3c27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4243b04b-76e1-4efe-8985-4f0cbe3b3c27" (UID: "4243b04b-76e1-4efe-8985-4f0cbe3b3c27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:20:49.330262 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.330195    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4243b04b-76e1-4efe-8985-4f0cbe3b3c27-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:20:49.989098 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.989070    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"
Sep 18 19:20:49.989265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.989069    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd" event={"ID":"4243b04b-76e1-4efe-8985-4f0cbe3b3c27","Type":"ContainerDied","Data":"9b566a7a6d6bc46e6f570dd0d8e9e11c92bbbbe4746a2a4caa75d1afb0984947"}
Sep 18 19:20:49.989265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:49.989195    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b566a7a6d6bc46e6f570dd0d8e9e11c92bbbbe4746a2a4caa75d1afb0984947"
Sep 18 19:20:59.108398 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:59.108366    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"]
Sep 18 19:20:59.131309 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:20:59.131283    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8f9642-2b2xd"]
Sep 18 19:21:01.054434 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:01.054404    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4243b04b-76e1-4efe-8985-4f0cbe3b3c27" path="/var/lib/kubelet/pods/4243b04b-76e1-4efe-8985-4f0cbe3b3c27/volumes"
Sep 18 19:21:10.972799 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.972765    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"]
Sep 18 19:21:10.973172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.973164    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4243b04b-76e1-4efe-8985-4f0cbe3b3c27" containerName="revoke-keys"
Sep 18 19:21:10.973221 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.973175    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4243b04b-76e1-4efe-8985-4f0cbe3b3c27" containerName="revoke-keys"
Sep 18 19:21:10.973221 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.973198    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy"
Sep 18 19:21:10.973221 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.973203    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b2587b-2c04-4c53-a7c5-aac31f50c0b0" containerName="istio-proxy"
Sep 18 19:21:10.973315 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.973257    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4243b04b-76e1-4efe-8985-4f0cbe3b3c27" containerName="revoke-keys"
Sep 18 19:21:10.975601 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.975586    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:10.989425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.989406    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:21:10.993025 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:10.993011    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:21:11.099887 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:11.099852    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"]
Sep 18 19:21:11.137967 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:11.137941    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c4785a-4412-4b9e-8c33-37ac34548722-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d\" (UID: \"c3c4785a-4412-4b9e-8c33-37ac34548722\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:11.239344 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:11.239275    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c4785a-4412-4b9e-8c33-37ac34548722-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d\" (UID: \"c3c4785a-4412-4b9e-8c33-37ac34548722\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:11.239943 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:11.239924    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c4785a-4412-4b9e-8c33-37ac34548722-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d\" (UID: \"c3c4785a-4412-4b9e-8c33-37ac34548722\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:11.285450 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:11.285424    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:11.643627 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:11.643599    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"]
Sep 18 19:21:11.644864 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:21:11.644839    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c4785a_4412_4b9e_8c33_37ac34548722.slice/crio-200548885d950cc4d71879aa6924bf3d381f1b7f641e262153dda597bc19953f WatchSource:0}: Error finding container 200548885d950cc4d71879aa6924bf3d381f1b7f641e262153dda597bc19953f: Status 404 returned error can't find the container with id 200548885d950cc4d71879aa6924bf3d381f1b7f641e262153dda597bc19953f
Sep 18 19:21:12.101100 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:12.101062    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d" event={"ID":"c3c4785a-4412-4b9e-8c33-37ac34548722","Type":"ContainerStarted","Data":"74716554109dd2946f778116ce28c6e75e8125d998b36876b5c0aa5c97411928"}
Sep 18 19:21:12.101457 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:12.101124    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d" event={"ID":"c3c4785a-4412-4b9e-8c33-37ac34548722","Type":"ContainerStarted","Data":"200548885d950cc4d71879aa6924bf3d381f1b7f641e262153dda597bc19953f"}
Sep 18 19:21:12.156363 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:12.156303    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d" podStartSLOduration=2.156286659 podStartE2EDuration="2.156286659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:21:12.15539728 +0000 UTC m=+2201.593657484" watchObservedRunningTime="2026-09-18 19:21:12.156286659 +0000 UTC m=+2201.594546862"
Sep 18 19:21:13.107632 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:13.107593    2568 generic.go:356] "Generic (PLEG): container finished" podID="c3c4785a-4412-4b9e-8c33-37ac34548722" containerID="74716554109dd2946f778116ce28c6e75e8125d998b36876b5c0aa5c97411928" exitCode=0
Sep 18 19:21:13.108070 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:13.107696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d" event={"ID":"c3c4785a-4412-4b9e-8c33-37ac34548722","Type":"ContainerDied","Data":"74716554109dd2946f778116ce28c6e75e8125d998b36876b5c0aa5c97411928"}
Sep 18 19:21:14.248820 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:14.248796    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:14.268811 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:14.268789    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c4785a-4412-4b9e-8c33-37ac34548722-maas-api-service-ca\") pod \"c3c4785a-4412-4b9e-8c33-37ac34548722\" (UID: \"c3c4785a-4412-4b9e-8c33-37ac34548722\") "
Sep 18 19:21:14.269095 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:14.269077    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3c4785a-4412-4b9e-8c33-37ac34548722-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3c4785a-4412-4b9e-8c33-37ac34548722" (UID: "c3c4785a-4412-4b9e-8c33-37ac34548722"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:14.370538 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:14.370433    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c4785a-4412-4b9e-8c33-37ac34548722-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:15.120044 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:15.120008    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d" event={"ID":"c3c4785a-4412-4b9e-8c33-37ac34548722","Type":"ContainerDied","Data":"200548885d950cc4d71879aa6924bf3d381f1b7f641e262153dda597bc19953f"}
Sep 18 19:21:15.120196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:15.120051    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"
Sep 18 19:21:15.120196 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:15.120060    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="200548885d950cc4d71879aa6924bf3d381f1b7f641e262153dda597bc19953f"
Sep 18 19:21:21.785144 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:21.785107    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"]
Sep 18 19:21:21.785674 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:21.785324    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc" podUID="1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" containerName="payload-processing" containerID="cri-o://e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964" gracePeriod=30
Sep 18 19:21:21.802050 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:21.802027    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"]
Sep 18 19:21:21.802215 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:21.802196    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh" podUID="ca3301d4-6b12-4192-b8c7-b1c93372b0a1" containerName="payload-pre-processing" containerID="cri-o://e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607" gracePeriod=30
Sep 18 19:21:22.077729 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.077705    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:21:22.080740 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.080720    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:21:22.126023 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.125994    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"]
Sep 18 19:21:22.139024 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.138997    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42bt2\" (UniqueName: \"kubernetes.io/projected/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-kube-api-access-42bt2\") pod \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") "
Sep 18 19:21:22.139174 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.139153    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8pfp\" (UniqueName: \"kubernetes.io/projected/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-kube-api-access-f8pfp\") pod \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") "
Sep 18 19:21:22.139255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.139192    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-plugins-config-volume\") pod \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\" (UID: \"ca3301d4-6b12-4192-b8c7-b1c93372b0a1\") "
Sep 18 19:21:22.139255 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.139236    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-plugins-config-volume\") pod \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\" (UID: \"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7\") "
Sep 18 19:21:22.139674 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.139635    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca3301d4-6b12-4192-b8c7-b1c93372b0a1" (UID: "ca3301d4-6b12-4192-b8c7-b1c93372b0a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:22.139792 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.139687    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" (UID: "1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:22.141173 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.141147    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-feb80d-5pc9d"]
Sep 18 19:21:22.141605 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.141570    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-kube-api-access-f8pfp" (OuterVolumeSpecName: "kube-api-access-f8pfp") pod "1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" (UID: "1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7"). InnerVolumeSpecName "kube-api-access-f8pfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:22.141732 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.141706    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-kube-api-access-42bt2" (OuterVolumeSpecName: "kube-api-access-42bt2") pod "ca3301d4-6b12-4192-b8c7-b1c93372b0a1" (UID: "ca3301d4-6b12-4192-b8c7-b1c93372b0a1"). InnerVolumeSpecName "kube-api-access-42bt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:22.159981 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.158863    2568 generic.go:356] "Generic (PLEG): container finished" podID="1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" containerID="e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964" exitCode=0
Sep 18 19:21:22.159981 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.159031    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc" event={"ID":"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7","Type":"ContainerDied","Data":"e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964"}
Sep 18 19:21:22.159981 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.159082    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc" event={"ID":"1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7","Type":"ContainerDied","Data":"fb71bb7ede8dfe0e1df70b0e9aa33b9eb48f0227db2c4fafd449064b5389d690"}
Sep 18 19:21:22.159981 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.159112    2568 scope.go:117] "RemoveContainer" containerID="e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964"
Sep 18 19:21:22.159981 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.159274    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"
Sep 18 19:21:22.163329 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.163145    2568 generic.go:356] "Generic (PLEG): container finished" podID="ca3301d4-6b12-4192-b8c7-b1c93372b0a1" containerID="e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607" exitCode=0
Sep 18 19:21:22.163329 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.163242    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh" event={"ID":"ca3301d4-6b12-4192-b8c7-b1c93372b0a1","Type":"ContainerDied","Data":"e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607"}
Sep 18 19:21:22.163329 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.163285    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh" event={"ID":"ca3301d4-6b12-4192-b8c7-b1c93372b0a1","Type":"ContainerDied","Data":"40109a440e092d08fe7e832e878ce2ed359c0096ae939cdc878a50542276fc1b"}
Sep 18 19:21:22.164211 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.163848    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"
Sep 18 19:21:22.178079 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.178060    2568 scope.go:117] "RemoveContainer" containerID="e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964"
Sep 18 19:21:22.178430 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:21:22.178403    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964\": container with ID starting with e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964 not found: ID does not exist" containerID="e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964"
Sep 18 19:21:22.178563 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.178441    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964"} err="failed to get container status \"e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964\": rpc error: code = NotFound desc = could not find container \"e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964\": container with ID starting with e99f9f8246920ae2aeeb2fb2cad94edec2f0d201af957da47576de728f5bc964 not found: ID does not exist"
Sep 18 19:21:22.178563 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.178467    2568 scope.go:117] "RemoveContainer" containerID="e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607"
Sep 18 19:21:22.188050 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.188030    2568 scope.go:117] "RemoveContainer" containerID="e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607"
Sep 18 19:21:22.188331 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:21:22.188301    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607\": container with ID starting with e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607 not found: ID does not exist" containerID="e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607"
Sep 18 19:21:22.188412 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.188339    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607"} err="failed to get container status \"e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607\": rpc error: code = NotFound desc = could not find container \"e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607\": container with ID starting with e250db7bf492eee1cdb2e040d78cf62bf1b09ed32106cf1b3886cacf2f23c607 not found: ID does not exist"
Sep 18 19:21:22.209030 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.209004    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"]
Sep 18 19:21:22.210924 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.210905    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-feb80d-8567887949-snvdc"]
Sep 18 19:21:22.228537 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.228486    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"]
Sep 18 19:21:22.234936 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.234913    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-feb80d-7d8b8468c4-mcxrh"]
Sep 18 19:21:22.240802 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.240783    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8pfp\" (UniqueName: \"kubernetes.io/projected/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-kube-api-access-f8pfp\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:22.240861 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.240805    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:22.240861 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.240815    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:22.240861 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:22.240823    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42bt2\" (UniqueName: \"kubernetes.io/projected/ca3301d4-6b12-4192-b8c7-b1c93372b0a1-kube-api-access-42bt2\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:23.054069 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:23.054037    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" path="/var/lib/kubelet/pods/1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7/volumes"
Sep 18 19:21:23.054420 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:23.054343    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3c4785a-4412-4b9e-8c33-37ac34548722" path="/var/lib/kubelet/pods/c3c4785a-4412-4b9e-8c33-37ac34548722/volumes"
Sep 18 19:21:23.054681 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:23.054667    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca3301d4-6b12-4192-b8c7-b1c93372b0a1" path="/var/lib/kubelet/pods/ca3301d4-6b12-4192-b8c7-b1c93372b0a1/volumes"
Sep 18 19:21:31.055575 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.055548    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"]
Sep 18 19:21:31.055981 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.055732    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy" containerID="cri-o://b32e84103188a891a017a85d217bf5e6d35b00bff6fb3d8468ddaff19b6f7f40" gracePeriod=30
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.383579    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"]
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384426    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca3301d4-6b12-4192-b8c7-b1c93372b0a1" containerName="payload-pre-processing"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384446    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca3301d4-6b12-4192-b8c7-b1c93372b0a1" containerName="payload-pre-processing"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384495    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" containerName="payload-processing"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384504    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" containerName="payload-processing"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384542    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c4785a-4412-4b9e-8c33-37ac34548722" containerName="revoke-keys"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384551    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c4785a-4412-4b9e-8c33-37ac34548722" containerName="revoke-keys"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384737    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc9fd3e-4b80-4f56-ba51-36e8a8c85bb7" containerName="payload-processing"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384755    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca3301d4-6b12-4192-b8c7-b1c93372b0a1" containerName="payload-pre-processing"
Sep 18 19:21:31.386623 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.384764    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c4785a-4412-4b9e-8c33-37ac34548722" containerName="revoke-keys"
Sep 18 19:21:31.389180 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.389158    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:31.391910 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.391890    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 19:21:31.392338 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.392318    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kgn8b\""
Sep 18 19:21:31.396849 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.396828    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"]
Sep 18 19:21:31.426686 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.426664    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/805e16bd-ca19-4dd6-a5ed-15dd372a6032-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv\" (UID: \"805e16bd-ca19-4dd6-a5ed-15dd372a6032\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:31.527951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.527923    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/805e16bd-ca19-4dd6-a5ed-15dd372a6032-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv\" (UID: \"805e16bd-ca19-4dd6-a5ed-15dd372a6032\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:31.528478 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.528457    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/805e16bd-ca19-4dd6-a5ed-15dd372a6032-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv\" (UID: \"805e16bd-ca19-4dd6-a5ed-15dd372a6032\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:31.702025 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.702004    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:31.990590 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:31.990537    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"]
Sep 18 19:21:31.993502 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:21:31.993478    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805e16bd_ca19_4dd6_a5ed_15dd372a6032.slice/crio-636d715ec85ec838ae419bc712d20e994d53d0efbd04ed24f09647df49b3db56 WatchSource:0}: Error finding container 636d715ec85ec838ae419bc712d20e994d53d0efbd04ed24f09647df49b3db56: Status 404 returned error can't find the container with id 636d715ec85ec838ae419bc712d20e994d53d0efbd04ed24f09647df49b3db56
Sep 18 19:21:32.226372 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:32.226331    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv" event={"ID":"805e16bd-ca19-4dd6-a5ed-15dd372a6032","Type":"ContainerStarted","Data":"636d715ec85ec838ae419bc712d20e994d53d0efbd04ed24f09647df49b3db56"}
Sep 18 19:21:33.232498 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:33.232464    2568 generic.go:356] "Generic (PLEG): container finished" podID="805e16bd-ca19-4dd6-a5ed-15dd372a6032" containerID="9585aa48e1c9e4b37f30f0e54324e30ce316efeaa74f72985ac45006c8265437" exitCode=0
Sep 18 19:21:33.232908 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:33.232557    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv" event={"ID":"805e16bd-ca19-4dd6-a5ed-15dd372a6032","Type":"ContainerDied","Data":"9585aa48e1c9e4b37f30f0e54324e30ce316efeaa74f72985ac45006c8265437"}
Sep 18 19:21:34.369253 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:34.369229    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:34.455678 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:34.455651    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/805e16bd-ca19-4dd6-a5ed-15dd372a6032-maas-api-service-ca\") pod \"805e16bd-ca19-4dd6-a5ed-15dd372a6032\" (UID: \"805e16bd-ca19-4dd6-a5ed-15dd372a6032\") "
Sep 18 19:21:34.455980 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:34.455961    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805e16bd-ca19-4dd6-a5ed-15dd372a6032-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "805e16bd-ca19-4dd6-a5ed-15dd372a6032" (UID: "805e16bd-ca19-4dd6-a5ed-15dd372a6032"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:34.557165 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:34.557096    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/805e16bd-ca19-4dd6-a5ed-15dd372a6032-maas-api-service-ca\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:35.243575 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:35.243543    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv" event={"ID":"805e16bd-ca19-4dd6-a5ed-15dd372a6032","Type":"ContainerDied","Data":"636d715ec85ec838ae419bc712d20e994d53d0efbd04ed24f09647df49b3db56"}
Sep 18 19:21:35.243728 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:35.243598    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="636d715ec85ec838ae419bc712d20e994d53d0efbd04ed24f09647df49b3db56"
Sep 18 19:21:35.243728 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:35.243558    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"
Sep 18 19:21:36.251852 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.251820    2568 generic.go:356] "Generic (PLEG): container finished" podID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerID="b32e84103188a891a017a85d217bf5e6d35b00bff6fb3d8468ddaff19b6f7f40" exitCode=0
Sep 18 19:21:36.252272 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.251886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" event={"ID":"c76fd339-cc47-4f20-9997-6ae985bc4d03","Type":"ContainerDied","Data":"b32e84103188a891a017a85d217bf5e6d35b00bff6fb3d8468ddaff19b6f7f40"}
Sep 18 19:21:36.252272 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.251964    2568 scope.go:117] "RemoveContainer" containerID="c2f5349beed3e24746a0ce62b7120ea3637f495a459ae33bc233de5a4abf10db"
Sep 18 19:21:36.305122 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.305103    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:21:36.372999 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.372917    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-socket\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.372999 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.372956    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-credential-socket\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.372999 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.372994    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-envoy\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373017    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c76fd339-cc47-4f20-9997-6ae985bc4d03-istiod-ca-cert\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373045    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hfqj\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-kube-api-access-8hfqj\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373069    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-token\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373096    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-podinfo\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373165    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-data\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373203    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-certs\") pod \"c76fd339-cc47-4f20-9997-6ae985bc4d03\" (UID: \"c76fd339-cc47-4f20-9997-6ae985bc4d03\") "
Sep 18 19:21:36.373260 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373210    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:36.373641 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373345    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:36.373641 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373436    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76fd339-cc47-4f20-9997-6ae985bc4d03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:36.373641 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373612    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-data" (OuterVolumeSpecName: "istio-data") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:36.373641 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373624    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:36.373841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373668    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.373841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373688    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-credential-socket\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.373841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.373702    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c76fd339-cc47-4f20-9997-6ae985bc4d03-istiod-ca-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.375335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.375306    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:21:36.375335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.375314    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-token" (OuterVolumeSpecName: "istio-token") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:36.375479 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.375353    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-kube-api-access-8hfqj" (OuterVolumeSpecName: "kube-api-access-8hfqj") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "kube-api-access-8hfqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:36.375479 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.375379    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c76fd339-cc47-4f20-9997-6ae985bc4d03" (UID: "c76fd339-cc47-4f20-9997-6ae985bc4d03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 19:21:36.474296 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.474271    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-envoy\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.474296 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.474295    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hfqj\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-kube-api-access-8hfqj\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.474462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.474304    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-token\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.474462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.474313    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-podinfo\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.474462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.474320    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-istio-data\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:36.474462 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:36.474327    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c76fd339-cc47-4f20-9997-6ae985bc4d03-workload-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:37.258813 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:37.258789    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"
Sep 18 19:21:37.259172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:37.258788    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb" event={"ID":"c76fd339-cc47-4f20-9997-6ae985bc4d03","Type":"ContainerDied","Data":"c82840e146e6b17c6676b583b0422b5f75ba8a8280c6b0c69251b98969e1be12"}
Sep 18 19:21:37.259172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:37.258913    2568 scope.go:117] "RemoveContainer" containerID="b32e84103188a891a017a85d217bf5e6d35b00bff6fb3d8468ddaff19b6f7f40"
Sep 18 19:21:37.285081 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:37.285055    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"]
Sep 18 19:21:37.292356 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:37.292333    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-feb80d-openshift-default-6ffffdfc8-g52xb"]
Sep 18 19:21:39.054301 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:39.054273    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" path="/var/lib/kubelet/pods/c76fd339-cc47-4f20-9997-6ae985bc4d03/volumes"
Sep 18 19:21:42.493197 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.493162    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"]
Sep 18 19:21:42.501223 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.501193    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-19944d-29829315-2j7wn"]
Sep 18 19:21:42.603746 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.603716    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"]
Sep 18 19:21:42.603950 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.603913    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd" podUID="23ec8680-419d-4e3f-b1db-6e2fe9988ef1" containerName="payload-processing" containerID="cri-o://d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a" gracePeriod=30
Sep 18 19:21:42.616200 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.616177    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"]
Sep 18 19:21:42.616401 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.616381    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf" podUID="d68a0165-d103-46b9-aee1-0e8cfd099e66" containerName="payload-pre-processing" containerID="cri-o://96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680" gracePeriod=30
Sep 18 19:21:42.870950 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.870925    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:21:42.896287 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:42.896263    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:21:43.037645 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.037567    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qktz\" (UniqueName: \"kubernetes.io/projected/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-kube-api-access-5qktz\") pod \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") "
Sep 18 19:21:43.037645 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.037616    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-plugins-config-volume\") pod \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\" (UID: \"23ec8680-419d-4e3f-b1db-6e2fe9988ef1\") "
Sep 18 19:21:43.037875 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.037668    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68a0165-d103-46b9-aee1-0e8cfd099e66-plugins-config-volume\") pod \"d68a0165-d103-46b9-aee1-0e8cfd099e66\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") "
Sep 18 19:21:43.037875 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.037726    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nddg\" (UniqueName: \"kubernetes.io/projected/d68a0165-d103-46b9-aee1-0e8cfd099e66-kube-api-access-8nddg\") pod \"d68a0165-d103-46b9-aee1-0e8cfd099e66\" (UID: \"d68a0165-d103-46b9-aee1-0e8cfd099e66\") "
Sep 18 19:21:43.038094 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.038070    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23ec8680-419d-4e3f-b1db-6e2fe9988ef1" (UID: "23ec8680-419d-4e3f-b1db-6e2fe9988ef1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:43.038172 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.038073    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68a0165-d103-46b9-aee1-0e8cfd099e66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d68a0165-d103-46b9-aee1-0e8cfd099e66" (UID: "d68a0165-d103-46b9-aee1-0e8cfd099e66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:21:43.039833 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.039811    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68a0165-d103-46b9-aee1-0e8cfd099e66-kube-api-access-8nddg" (OuterVolumeSpecName: "kube-api-access-8nddg") pod "d68a0165-d103-46b9-aee1-0e8cfd099e66" (UID: "d68a0165-d103-46b9-aee1-0e8cfd099e66"). InnerVolumeSpecName "kube-api-access-8nddg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:43.039942 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.039876    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-kube-api-access-5qktz" (OuterVolumeSpecName: "kube-api-access-5qktz") pod "23ec8680-419d-4e3f-b1db-6e2fe9988ef1" (UID: "23ec8680-419d-4e3f-b1db-6e2fe9988ef1"). InnerVolumeSpecName "kube-api-access-5qktz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:21:43.054335 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.054307    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72fa953-ef41-4604-b92f-dcc4510a742c" path="/var/lib/kubelet/pods/b72fa953-ef41-4604-b92f-dcc4510a742c/volumes"
Sep 18 19:21:43.076060 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.076037    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"]
Sep 18 19:21:43.090923 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.090900    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-19944d-xz7zv"]
Sep 18 19:21:43.139497 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.139468    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nddg\" (UniqueName: \"kubernetes.io/projected/d68a0165-d103-46b9-aee1-0e8cfd099e66-kube-api-access-8nddg\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:43.139497 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.139490    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qktz\" (UniqueName: \"kubernetes.io/projected/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-kube-api-access-5qktz\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:43.139497 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.139501    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ec8680-419d-4e3f-b1db-6e2fe9988ef1-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:43.139761 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.139510    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68a0165-d103-46b9-aee1-0e8cfd099e66-plugins-config-volume\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:21:43.292261 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.292174    2568 generic.go:356] "Generic (PLEG): container finished" podID="23ec8680-419d-4e3f-b1db-6e2fe9988ef1" containerID="d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a" exitCode=0
Sep 18 19:21:43.292261 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.292216    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"
Sep 18 19:21:43.292487 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.292258    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd" event={"ID":"23ec8680-419d-4e3f-b1db-6e2fe9988ef1","Type":"ContainerDied","Data":"d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a"}
Sep 18 19:21:43.292487 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.292323    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd" event={"ID":"23ec8680-419d-4e3f-b1db-6e2fe9988ef1","Type":"ContainerDied","Data":"392aa7af84d0b54767d2fcf847a4fbb5bae8a581d9bc3fdcfb8e14eaac5a2f6a"}
Sep 18 19:21:43.292487 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.292351    2568 scope.go:117] "RemoveContainer" containerID="d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a"
Sep 18 19:21:43.293605 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.293582    2568 generic.go:356] "Generic (PLEG): container finished" podID="d68a0165-d103-46b9-aee1-0e8cfd099e66" containerID="96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680" exitCode=0
Sep 18 19:21:43.293701 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.293632    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"
Sep 18 19:21:43.293759 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.293631    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf" event={"ID":"d68a0165-d103-46b9-aee1-0e8cfd099e66","Type":"ContainerDied","Data":"96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680"}
Sep 18 19:21:43.293815 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.293763    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf" event={"ID":"d68a0165-d103-46b9-aee1-0e8cfd099e66","Type":"ContainerDied","Data":"4549f903cb0021e8af8a9a8906eb05c67ff167a13b1f256285100185377cf15c"}
Sep 18 19:21:43.300918 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.300903    2568 scope.go:117] "RemoveContainer" containerID="d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a"
Sep 18 19:21:43.301177 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:21:43.301159    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a\": container with ID starting with d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a not found: ID does not exist" containerID="d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a"
Sep 18 19:21:43.301249 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.301187    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a"} err="failed to get container status \"d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a\": rpc error: code = NotFound desc = could not find container \"d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a\": container with ID starting with d25bda3de5e328437c1c09246bb14e3483a6e26d1758b459a709eb0410f84b1a not found: ID does not exist"
Sep 18 19:21:43.301249 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.301209    2568 scope.go:117] "RemoveContainer" containerID="96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680"
Sep 18 19:21:43.308567 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.308549    2568 scope.go:117] "RemoveContainer" containerID="96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680"
Sep 18 19:21:43.308822 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:21:43.308806    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680\": container with ID starting with 96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680 not found: ID does not exist" containerID="96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680"
Sep 18 19:21:43.308882 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.308829    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680"} err="failed to get container status \"96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680\": rpc error: code = NotFound desc = could not find container \"96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680\": container with ID starting with 96487f0ab92123d74f538237c0299ac1680c9cbc7c01fa06e8ee0f7a907fd680 not found: ID does not exist"
Sep 18 19:21:43.332087 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.332063    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"]
Sep 18 19:21:43.339287 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.339260    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-19944d-7df8bf869d-sd6vf"]
Sep 18 19:21:43.367680 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.367657    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"]
Sep 18 19:21:43.379259 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:43.379234    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-19944d-6865d7bbf5-9mhfd"]
Sep 18 19:21:45.054336 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:45.054302    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23ec8680-419d-4e3f-b1db-6e2fe9988ef1" path="/var/lib/kubelet/pods/23ec8680-419d-4e3f-b1db-6e2fe9988ef1/volumes"
Sep 18 19:21:45.054719 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:45.054670    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805e16bd-ca19-4dd6-a5ed-15dd372a6032" path="/var/lib/kubelet/pods/805e16bd-ca19-4dd6-a5ed-15dd372a6032/volumes"
Sep 18 19:21:45.054985 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:21:45.054972    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d68a0165-d103-46b9-aee1-0e8cfd099e66" path="/var/lib/kubelet/pods/d68a0165-d103-46b9-aee1-0e8cfd099e66/volumes"
Sep 18 19:22:31.647489 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:22:31.647462    2568 scope.go:117] "RemoveContainer" containerID="8e2ecc41792f56b2d7d6cb0ec488ede6b4d87da4360f907d245965b49c4978c7"
Sep 18 19:23:01.215862 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.215827    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84944d87d7-m24t6"]
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216209    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216220    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216230    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68a0165-d103-46b9-aee1-0e8cfd099e66" containerName="payload-pre-processing"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216236    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68a0165-d103-46b9-aee1-0e8cfd099e66" containerName="payload-pre-processing"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216257    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216263    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216275    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805e16bd-ca19-4dd6-a5ed-15dd372a6032" containerName="revoke-keys"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216280    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="805e16bd-ca19-4dd6-a5ed-15dd372a6032" containerName="revoke-keys"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216287    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23ec8680-419d-4e3f-b1db-6e2fe9988ef1" containerName="payload-processing"
Sep 18 19:23:01.216295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216292    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="23ec8680-419d-4e3f-b1db-6e2fe9988ef1" containerName="payload-processing"
Sep 18 19:23:01.216633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216355    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68a0165-d103-46b9-aee1-0e8cfd099e66" containerName="payload-pre-processing"
Sep 18 19:23:01.216633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216364    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy"
Sep 18 19:23:01.216633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216369    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="23ec8680-419d-4e3f-b1db-6e2fe9988ef1" containerName="payload-processing"
Sep 18 19:23:01.216633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216375    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76fd339-cc47-4f20-9997-6ae985bc4d03" containerName="istio-proxy"
Sep 18 19:23:01.216633 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.216380    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="805e16bd-ca19-4dd6-a5ed-15dd372a6032" containerName="revoke-keys"
Sep 18 19:23:01.219383 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.219368    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.223431 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.223402    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 19:23:01.223571 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.223432    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pcg7z\""
Sep 18 19:23:01.223571 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.223468    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 19:23:01.223689 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.223596    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 19:23:01.239477 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.239446    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84944d87d7-m24t6"]
Sep 18 19:23:01.370119 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.370084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-metrics-certs\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.370119 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.370125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-webhook-cert\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.370318 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.370161    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fjbw\" (UniqueName: \"kubernetes.io/projected/7d91dcaa-c797-483b-b84c-c171529b6082-kube-api-access-9fjbw\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.471373 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.471297    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fjbw\" (UniqueName: \"kubernetes.io/projected/7d91dcaa-c797-483b-b84c-c171529b6082-kube-api-access-9fjbw\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.471562 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.471383    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-metrics-certs\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.471562 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.471411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-webhook-cert\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.473926 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.473904    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-metrics-certs\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.474032 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.473956    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-webhook-cert\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.481256 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.481229    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fjbw\" (UniqueName: \"kubernetes.io/projected/7d91dcaa-c797-483b-b84c-c171529b6082-kube-api-access-9fjbw\") pod \"maas-controller-84944d87d7-m24t6\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") " pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.530683 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.530651    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:01.807599 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.807574    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84944d87d7-m24t6"]
Sep 18 19:23:01.809074 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:23:01.809052    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d91dcaa_c797_483b_b84c_c171529b6082.slice/crio-2a6813b2f676533fe62a59f8c2d5ae7d1ea6d4408618489ca8573b4cf5751e83 WatchSource:0}: Error finding container 2a6813b2f676533fe62a59f8c2d5ae7d1ea6d4408618489ca8573b4cf5751e83: Status 404 returned error can't find the container with id 2a6813b2f676533fe62a59f8c2d5ae7d1ea6d4408618489ca8573b4cf5751e83
Sep 18 19:23:01.810471 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:01.810455    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:23:02.689913 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:02.689814    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-m24t6" event={"ID":"7d91dcaa-c797-483b-b84c-c171529b6082","Type":"ContainerStarted","Data":"84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42"}
Sep 18 19:23:02.689913 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:02.689876    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-m24t6" event={"ID":"7d91dcaa-c797-483b-b84c-c171529b6082","Type":"ContainerStarted","Data":"2a6813b2f676533fe62a59f8c2d5ae7d1ea6d4408618489ca8573b4cf5751e83"}
Sep 18 19:23:02.690295 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:02.689941    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:02.721451 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:02.721404    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84944d87d7-m24t6" podStartSLOduration=1.246642615 podStartE2EDuration="1.721388817s" totalImagesPullingTime="474.746202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:23:02.721044916 +0000 UTC m=+2312.159305159" watchObservedRunningTime="2026-09-18 19:23:02.721388817 +0000 UTC m=+2312.159649021"
Sep 18 19:23:13.696659 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:13.696626    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:23.103777 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.103742    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84944d87d7-m24t6"]
Sep 18 19:23:23.104270 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.103941    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84944d87d7-m24t6" podUID="7d91dcaa-c797-483b-b84c-c171529b6082" containerName="manager" containerID="cri-o://84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42" gracePeriod=30
Sep 18 19:23:23.358727 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.358657    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:23.486111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.486019    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-metrics-certs\") pod \"7d91dcaa-c797-483b-b84c-c171529b6082\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") "
Sep 18 19:23:23.486111 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.486067    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fjbw\" (UniqueName: \"kubernetes.io/projected/7d91dcaa-c797-483b-b84c-c171529b6082-kube-api-access-9fjbw\") pod \"7d91dcaa-c797-483b-b84c-c171529b6082\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") "
Sep 18 19:23:23.486378 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.486209    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-webhook-cert\") pod \"7d91dcaa-c797-483b-b84c-c171529b6082\" (UID: \"7d91dcaa-c797-483b-b84c-c171529b6082\") "
Sep 18 19:23:23.488297 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.488264    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d91dcaa-c797-483b-b84c-c171529b6082" (UID: "7d91dcaa-c797-483b-b84c-c171529b6082"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:23:23.488430 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.488315    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d91dcaa-c797-483b-b84c-c171529b6082-kube-api-access-9fjbw" (OuterVolumeSpecName: "kube-api-access-9fjbw") pod "7d91dcaa-c797-483b-b84c-c171529b6082" (UID: "7d91dcaa-c797-483b-b84c-c171529b6082"). InnerVolumeSpecName "kube-api-access-9fjbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:23:23.488430 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.488355    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7d91dcaa-c797-483b-b84c-c171529b6082" (UID: "7d91dcaa-c797-483b-b84c-c171529b6082"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:23:23.587710 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.587677    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-webhook-cert\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:23:23.587710 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.587707    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d91dcaa-c797-483b-b84c-c171529b6082-metrics-certs\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:23:23.587710 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.587717    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fjbw\" (UniqueName: \"kubernetes.io/projected/7d91dcaa-c797-483b-b84c-c171529b6082-kube-api-access-9fjbw\") on node \"ip-10-0-132-94.ec2.internal\" DevicePath \"\""
Sep 18 19:23:23.795402 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.795363    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d91dcaa-c797-483b-b84c-c171529b6082" containerID="84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42" exitCode=0
Sep 18 19:23:23.795609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.795413    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84944d87d7-m24t6"
Sep 18 19:23:23.795609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.795407    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-m24t6" event={"ID":"7d91dcaa-c797-483b-b84c-c171529b6082","Type":"ContainerDied","Data":"84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42"}
Sep 18 19:23:23.795609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.795496    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84944d87d7-m24t6" event={"ID":"7d91dcaa-c797-483b-b84c-c171529b6082","Type":"ContainerDied","Data":"2a6813b2f676533fe62a59f8c2d5ae7d1ea6d4408618489ca8573b4cf5751e83"}
Sep 18 19:23:23.795609 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.795531    2568 scope.go:117] "RemoveContainer" containerID="84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42"
Sep 18 19:23:23.804841 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.804819    2568 scope.go:117] "RemoveContainer" containerID="84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42"
Sep 18 19:23:23.805106 ip-10-0-132-94 kubenswrapper[2568]: E0918 19:23:23.805088    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42\": container with ID starting with 84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42 not found: ID does not exist" containerID="84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42"
Sep 18 19:23:23.805156 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.805115    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42"} err="failed to get container status \"84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42\": rpc error: code = NotFound desc = could not find container \"84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42\": container with ID starting with 84a7f897db4764b4dc5cf0683e46740819522be12f4801df8eee7b08245d4d42 not found: ID does not exist"
Sep 18 19:23:23.822217 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.822184    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84944d87d7-m24t6"]
Sep 18 19:23:23.826452 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:23.826429    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84944d87d7-m24t6"]
Sep 18 19:23:25.060035 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:23:25.060002    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d91dcaa-c797-483b-b84c-c171529b6082" path="/var/lib/kubelet/pods/7d91dcaa-c797-483b-b84c-c171529b6082/volumes"
Sep 18 19:24:31.189750 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:24:31.189719    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:24:31.197072 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:24:31.197049    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:25:31.696119 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:25:31.696091    2568 scope.go:117] "RemoveContainer" containerID="aeb55b5f9fc5fa2e7b68609bd6cb5e9f7c6cc010d65b67844d5790eefc37813b"
Sep 18 19:26:31.715946 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:26:31.715905    2568 scope.go:117] "RemoveContainer" containerID="b7cfe3e9859c866a9b3b33bc072dc0b8d2d0d90e84ffe415605b3c2f08a2fe0f"
Sep 18 19:27:31.736477 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:27:31.736445    2568 scope.go:117] "RemoveContainer" containerID="74716554109dd2946f778116ce28c6e75e8125d998b36876b5c0aa5c97411928"
Sep 18 19:27:31.744710 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:27:31.744690    2568 scope.go:117] "RemoveContainer" containerID="04e562f26475715275a7376f7862386648b82ad0cea6770634982e24b88d723f"
Sep 18 19:28:19.599719 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:19.599688    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-prjmf_dc0f4ba6-c961-4010-ac60-4f834224caa5/server/0.log"
Sep 18 19:28:19.732399 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:19.732365    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7mjbt_8e629ed7-eab2-4cda-b3fd-0ede8a90f89a/manager/0.log"
Sep 18 19:28:19.875396 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:19.875318    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-6dqvz_2124a540-3613-467a-8373-4b124a547095/copy-manifests/0.log"
Sep 18 19:28:19.900110 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:19.900074    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-6dqvz_2124a540-3613-467a-8373-4b124a547095/manager/0.log"
Sep 18 19:28:20.974265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:20.974235    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29829315-wvbh2_4b6ff38c-3d77-47f2-9eb5-19bfd2e2e578/cleanup/0.log"
Sep 18 19:28:23.163288 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:23.163243    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk_88efd088-574b-4128-80c9-5554600d3baf/istio-proxy/0.log"
Sep 18 19:28:23.283555 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:23.283475    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tqd44_315991ff-9db3-466e-bb57-5708a6d66a33/discovery/0.log"
Sep 18 19:28:23.788713 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:23.788682    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kjc76_268b3d97-09f9-458d-9204-d460a6112881/payload-pre-processing/0.log"
Sep 18 19:28:24.057813 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:24.057686    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8587f4c686-59cps_85c147de-2421-41c8-b625-33d6a9cb1e58/router/0.log"
Sep 18 19:28:24.556090 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:24.556062    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-sq7ck_29b3b716-9bc2-4387-a522-cfe713ae0290/main/0.log"
Sep 18 19:28:24.669103 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:24.669077    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-q45rh_01ce02d2-ee0a-402d-898d-8ab61359389d/main/0.log"
Sep 18 19:28:24.909479 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:24.909400    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frn42g_ad654551-c382-4f6a-abe6-accbed87ab01/main/0.log"
Sep 18 19:28:31.766310 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:31.766277    2568 scope.go:117] "RemoveContainer" containerID="9585aa48e1c9e4b37f30f0e54324e30ce316efeaa74f72985ac45006c8265437"
Sep 18 19:28:35.109209 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:35.109174    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hq4lf_51fa9a7e-c852-4d5b-9fc4-6d75203eb0dd/global-pull-secret-syncer/0.log"
Sep 18 19:28:35.345141 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:35.345110    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nfq7b_d3e57d49-e5b9-454c-925c-24f923cf8c69/konnectivity-agent/0.log"
Sep 18 19:28:35.400558 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:35.400467    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-94.ec2.internal_8708fb89a1341e781b6f6cd3d321ee06/haproxy/0.log"
Sep 18 19:28:42.415465 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.415433    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-xh4mn_54c8cb6a-e9d6-4e73-bfa6-06c03557bed9/kube-state-metrics/0.log"
Sep 18 19:28:42.439316 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.439283    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-xh4mn_54c8cb6a-e9d6-4e73-bfa6-06c03557bed9/kube-rbac-proxy-main/0.log"
Sep 18 19:28:42.477176 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.477149    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-xh4mn_54c8cb6a-e9d6-4e73-bfa6-06c03557bed9/kube-rbac-proxy-self/0.log"
Sep 18 19:28:42.513400 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.513333    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b4745fdf5-dvks9_b51f082b-ac56-416a-8532-ae1c87a60e81/metrics-server/0.log"
Sep 18 19:28:42.550425 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.550396    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-d9bn9_66b48134-5307-4c8d-a9da-001c9fb0dd32/monitoring-plugin/0.log"
Sep 18 19:28:42.591911 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.591881    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwzgb_54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b/node-exporter/0.log"
Sep 18 19:28:42.630587 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.630559    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwzgb_54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b/kube-rbac-proxy/0.log"
Sep 18 19:28:42.657448 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:42.657425    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwzgb_54ce4f76-a83b-4ecd-8c65-6afc06ef5f1b/init-textfile/0.log"
Sep 18 19:28:43.046641 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.046615    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pxq2x_2f6632a5-2f4f-49b0-95c7-a84ff5480f76/kube-rbac-proxy-main/0.log"
Sep 18 19:28:43.097265 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.097226    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pxq2x_2f6632a5-2f4f-49b0-95c7-a84ff5480f76/kube-rbac-proxy-self/0.log"
Sep 18 19:28:43.163635 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.163603    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-pxq2x_2f6632a5-2f4f-49b0-95c7-a84ff5480f76/openshift-state-metrics/0.log"
Sep 18 19:28:43.245951 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.245908    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/prometheus/0.log"
Sep 18 19:28:43.307292 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.307216    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/config-reloader/0.log"
Sep 18 19:28:43.408525 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.408496    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/thanos-sidecar/0.log"
Sep 18 19:28:43.539972 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.539941    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/kube-rbac-proxy-web/0.log"
Sep 18 19:28:43.638289 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.638213    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/kube-rbac-proxy/0.log"
Sep 18 19:28:43.705781 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.705755    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/kube-rbac-proxy-thanos/0.log"
Sep 18 19:28:43.745816 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.745784    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d74a03b3-7907-4975-a792-c14f534599a9/init-config-reloader/0.log"
Sep 18 19:28:43.799383 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.799351    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-zlwsf_f63723ad-8bbf-4b77-8e17-775f4c503e7b/prometheus-operator/0.log"
Sep 18 19:28:43.833709 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:43.833676    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-zlwsf_f63723ad-8bbf-4b77-8e17-775f4c503e7b/kube-rbac-proxy/0.log"
Sep 18 19:28:44.457358 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.457322    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"]
Sep 18 19:28:44.457750 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.457737    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d91dcaa-c797-483b-b84c-c171529b6082" containerName="manager"
Sep 18 19:28:44.457803 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.457753    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d91dcaa-c797-483b-b84c-c171529b6082" containerName="manager"
Sep 18 19:28:44.457840 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.457833    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d91dcaa-c797-483b-b84c-c171529b6082" containerName="manager"
Sep 18 19:28:44.460938 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.460923    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.464793 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.464767    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qnj9f\"/\"kube-root-ca.crt\""
Sep 18 19:28:44.466309 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.466289    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qnj9f\"/\"openshift-service-ca.crt\""
Sep 18 19:28:44.467182 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.467159    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qnj9f\"/\"default-dockercfg-9tcdj\""
Sep 18 19:28:44.474655 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.474632    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"]
Sep 18 19:28:44.541683 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.541648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-podres\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.542070 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.541704    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-lib-modules\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.542070 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.541723    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zmwh\" (UniqueName: \"kubernetes.io/projected/3cdbdd34-692b-481f-bbc8-a5561f56265e-kube-api-access-8zmwh\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.542070 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.541810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-proc\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.542070 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.541845    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-sys\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.642900 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.642863    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-podres\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.642917    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-lib-modules\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643020    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zmwh\" (UniqueName: \"kubernetes.io/projected/3cdbdd34-692b-481f-bbc8-a5561f56265e-kube-api-access-8zmwh\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643031    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-podres\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643078 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-lib-modules\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643219 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643103    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-proc\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643219 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643133    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-sys\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643305 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643223    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-proc\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.643305 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.643298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cdbdd34-692b-481f-bbc8-a5561f56265e-sys\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.653569 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.653546    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zmwh\" (UniqueName: \"kubernetes.io/projected/3cdbdd34-692b-481f-bbc8-a5561f56265e-kube-api-access-8zmwh\") pod \"perf-node-gather-daemonset-6h5cf\" (UID: \"3cdbdd34-692b-481f-bbc8-a5561f56265e\") " pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:44.770546 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:44.770493    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:45.054633 ip-10-0-132-94 kubenswrapper[2568]: W0918 19:28:45.054599    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3cdbdd34_692b_481f_bbc8_a5561f56265e.slice/crio-e71e0fe35f9fb53c7006c16f8c6d06e3ea40131edf56990c493cdb6695de3a5a WatchSource:0}: Error finding container e71e0fe35f9fb53c7006c16f8c6d06e3ea40131edf56990c493cdb6695de3a5a: Status 404 returned error can't find the container with id e71e0fe35f9fb53c7006c16f8c6d06e3ea40131edf56990c493cdb6695de3a5a
Sep 18 19:28:45.055743 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:45.055722    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"]
Sep 18 19:28:45.394654 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:45.394539    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf" event={"ID":"3cdbdd34-692b-481f-bbc8-a5561f56265e","Type":"ContainerStarted","Data":"8eb3e7b38a0ed6dffdce4001ab583736927d832063d0235a281f0ccfc47b119d"}
Sep 18 19:28:45.394654 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:45.394599    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf" event={"ID":"3cdbdd34-692b-481f-bbc8-a5561f56265e","Type":"ContainerStarted","Data":"e71e0fe35f9fb53c7006c16f8c6d06e3ea40131edf56990c493cdb6695de3a5a"}
Sep 18 19:28:45.394921 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:45.394723    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:46.292284 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:46.292253    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/1.log"
Sep 18 19:28:46.296770 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:46.296748    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-dncfj_3125a1e7-a100-4427-b812-ca56cf0ef2be/console-operator/2.log"
Sep 18 19:28:46.926122 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:46.926090    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-h4xb8_ac032255-b81a-41bd-9a08-278e796f31c7/download-server/0.log"
Sep 18 19:28:49.127435 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:49.127409    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qgl2s_a48b6c68-2dc3-4f03-b60a-1b181429c33e/dns/0.log"
Sep 18 19:28:49.167854 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:49.167827    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qgl2s_a48b6c68-2dc3-4f03-b60a-1b181429c33e/kube-rbac-proxy/0.log"
Sep 18 19:28:49.338902 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:49.338875    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nsfwm_e8386b94-b143-4f08-a95a-ce6fb840c8f3/dns-node-resolver/0.log"
Sep 18 19:28:49.965308 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:49.965270    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-fcf6db547-s4hrj_f69e02dd-39f8-4394-8e29-63bf680f4817/registry/0.log"
Sep 18 19:28:50.065749 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:50.065719    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vdgtt_1f03eb80-0bb5-4a2a-b2a4-a5cfc0431fd4/node-ca/0.log"
Sep 18 19:28:51.077185 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:51.077152    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk_88efd088-574b-4128-80c9-5554600d3baf/istio-proxy/0.log"
Sep 18 19:28:51.410496 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:51.410418    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf"
Sep 18 19:28:51.437808 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:51.437752    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qnj9f/perf-node-gather-daemonset-6h5cf" podStartSLOduration=7.437735985 podStartE2EDuration="7.437735985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:28:45.423584819 +0000 UTC m=+2654.861845019" watchObservedRunningTime="2026-09-18 19:28:51.437735985 +0000 UTC m=+2660.875996190"
Sep 18 19:28:51.507919 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:51.507884    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tqd44_315991ff-9db3-466e-bb57-5708a6d66a33/discovery/0.log"
Sep 18 19:28:51.667268 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:51.667178    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kjc76_268b3d97-09f9-458d-9204-d460a6112881/payload-pre-processing/0.log"
Sep 18 19:28:51.751043 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:51.751008    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8587f4c686-59cps_85c147de-2421-41c8-b625-33d6a9cb1e58/router/0.log"
Sep 18 19:28:52.777086 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:52.777056    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfx72_33113daa-1aea-42d9-82bd-ecaa8d22cc9a/serve-healthcheck-canary/0.log"
Sep 18 19:28:53.531644 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:53.531612    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-b9nxp_0b078765-14df-41e5-af2c-08bd9f34612e/insights-operator/0.log"
Sep 18 19:28:53.533293 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:53.533274    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-b9nxp_0b078765-14df-41e5-af2c-08bd9f34612e/insights-operator/1.log"
Sep 18 19:28:53.788866 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:53.788831    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-72pdf_b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939/kube-rbac-proxy/0.log"
Sep 18 19:28:53.839779 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:53.839752    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-72pdf_b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939/exporter/0.log"
Sep 18 19:28:53.923319 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:53.923292    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-72pdf_b39bfa58-d06f-4ad4-bf2e-3cd93d3aa939/extractor/0.log"
Sep 18 19:28:54.973964 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:54.973934    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk_88efd088-574b-4128-80c9-5554600d3baf/istio-proxy/0.log"
Sep 18 19:28:55.399873 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:55.399788    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tqd44_315991ff-9db3-466e-bb57-5708a6d66a33/discovery/0.log"
Sep 18 19:28:55.482009 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:55.481977    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kjc76_268b3d97-09f9-458d-9204-d460a6112881/payload-pre-processing/0.log"
Sep 18 19:28:55.526015 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:55.525985    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8587f4c686-59cps_85c147de-2421-41c8-b625-33d6a9cb1e58/router/0.log"
Sep 18 19:28:57.015542 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.015494    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-bk2sk_88efd088-574b-4128-80c9-5554600d3baf/istio-proxy/0.log"
Sep 18 19:28:57.149855 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.149821    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-prjmf_dc0f4ba6-c961-4010-ac60-4f834224caa5/server/0.log"
Sep 18 19:28:57.285717 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.285628    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7mjbt_8e629ed7-eab2-4cda-b3fd-0ede8a90f89a/manager/0.log"
Sep 18 19:28:57.312277 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.312250    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-6dqvz_2124a540-3613-467a-8373-4b124a547095/manager/0.log"
Sep 18 19:28:57.337527 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.337499    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699fb7c78d-6dqvz_2124a540-3613-467a-8373-4b124a547095/copy-manifests/0.log"
Sep 18 19:28:57.518080 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.518048    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-tqd44_315991ff-9db3-466e-bb57-5708a6d66a33/discovery/0.log"
Sep 18 19:28:57.625592 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.625447    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kjc76_268b3d97-09f9-458d-9204-d460a6112881/payload-pre-processing/0.log"
Sep 18 19:28:57.676819 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:57.676790    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8587f4c686-59cps_85c147de-2421-41c8-b625-33d6a9cb1e58/router/0.log"
Sep 18 19:28:58.772398 ip-10-0-132-94 kubenswrapper[2568]: I0918 19:28:58.772373    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bf466d58c-ghhdd_997a9b51-8512-4a2f-b15d-6193dbd3ece6/manager/0.log"