Sep 18 17:58:13.259279 ip-10-0-135-209 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 17:58:13.259291 ip-10-0-135-209 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 17:58:13.259317 ip-10-0-135-209 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 17:58:13.259635 ip-10-0-135-209 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 17:58:23.415545 ip-10-0-135-209 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 17:58:23.415563 ip-10-0-135-209 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7f4f356ae5384abf85bbe7dad418fc3a --
Sep 18 18:00:15.027293 ip-10-0-135-209 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 18:00:18.679310 ip-10-0-135-209 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:00:18.679310 ip-10-0-135-209 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 18:00:18.679310 ip-10-0-135-209 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:00:18.679310 ip-10-0-135-209 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 18:00:18.679310 ip-10-0-135-209 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:00:18.680155 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.679399    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 18:00:18.683164 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683142    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:18.683164 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683161    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:18.683164 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683168    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683174    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683178    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683183    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683187    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683192    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683196    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683200    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683204    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683209    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683213    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683217    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683221    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683225    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683229    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683232    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683236    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683250    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683254    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:18.683381 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683258    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683262    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683266    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683270    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683274    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683277    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683281    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683286    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683289    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683293    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683298    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683302    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683310    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683315    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683319    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683324    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683329    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683333    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683339    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:18.684199 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683345    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683350    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683355    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683360    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683365    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683369    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683373    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683377    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683381    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683385    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683390    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683394    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683398    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683402    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683406    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683428    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683432    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683436    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683440    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683445    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683449    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:18.685011 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683453    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683457    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683461    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683465    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683470    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683476    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683480    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683486    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683490    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683494    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683498    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683502    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683506    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683511    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683515    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683519    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683523    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683527    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683533    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683537    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:18.685909 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683541    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683545    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683549    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683553    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683558    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683562    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683565    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683570    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683573    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683578    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683582    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683586    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683590    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683594    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683598    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683602    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683607    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.683611    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684766    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:18.686769 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684778    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684783    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684788    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684793    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684797    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684801    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684806    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684810    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684814    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684818    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684823    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684828    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684833    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684838    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684842    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684846    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684850    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684854    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684859    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684863    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:18.687608 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684867    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684871    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684875    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684880    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684885    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684889    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684894    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684898    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684905    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684911    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684916    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684921    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684925    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684930    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684934    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684939    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684944    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684949    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684953    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684957    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:18.688324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684962    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684966    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684971    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684977    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684982    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684988    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684993    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.684997    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685002    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685006    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685010    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685014    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685018    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685022    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685026    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685030    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685034    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685039    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685043    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685072    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:18.688865 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685076    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685084    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685090    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685094    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685098    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685103    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685106    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685111    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685115    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685119    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685123    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685127    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685131    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685136    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685140    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685144    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685148    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685152    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685157    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:18.689441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685161    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685168    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685174    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685179    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685184    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685189    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685193    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685198    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685201    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685206    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685211    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685215    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685220    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685224    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685228    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685232    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685236    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685240    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685244    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:18.689892 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685876    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685885    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685891    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685896    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685900    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685905    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685909    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685913    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685916    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685920    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685924    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685928    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685933    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685937    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685942    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685946    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685950    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685955    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685959    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685963    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:18.690616 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685967    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685971    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685976    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685980    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685985    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685989    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685993    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.685997    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686001    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686006    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686010    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686014    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686018    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686022    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686026    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686031    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686035    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686042    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686069    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:18.691121 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686075    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686079    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686083    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686088    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686093    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686098    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686103    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686107    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686112    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686116    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686120    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686124    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686128    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686135    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686140    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686147    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686154    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686160    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686165    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:18.691627 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686169    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686174    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686179    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686183    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686188    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686192    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686196    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686201    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686205    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686209    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686213    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686217    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686222    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686226    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686230    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686235    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686239    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686243    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686247    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686251    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:18.692149 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686256    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686260    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686264    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686268    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686273    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686279    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686286    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686290    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686294    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686298    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686302    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686307    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686311    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686315    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686319    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686323    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686327    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686331    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686335    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686339    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:18.692641 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.686344    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686451    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686502    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686512    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686521    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686528    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686533    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686540    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686546    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686551    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686556    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686561    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686566    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686572    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686578    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686583    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686588    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686592    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686598    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686604    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686608    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686613    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686618    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686623    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 18:00:18.693146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686629    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686634    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686639    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686644    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686649    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686653    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686658    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686663    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686668    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686674    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686679    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686683    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686689    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686694    2582 flags.go:64] FLAG: --enable-server="true"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686699    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686705    2582 flags.go:64] FLAG: --event-burst="100"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686711    2582 flags.go:64] FLAG: --event-qps="50"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686715    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686720    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686725    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686730    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686736    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686742    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686747    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686752    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 18:00:18.693744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686756    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686762    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686767    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686772    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686777    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686781    2582 flags.go:64] FLAG: --feature-gates=""
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686787    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686791    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686796    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686801    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686806    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686811    2582 flags.go:64] FLAG: --help="false"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686815    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-135-209.ec2.internal"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686820    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686825    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686830    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686835    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686841    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686845    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686851    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686856    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686860    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686865    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686870    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 18:00:18.694421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686875    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686879    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686884    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686890    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686895    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686899    2582 flags.go:64] FLAG: --lock-file=""
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686904    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686909    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686914    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686922    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686927    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686931    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686936    2582 flags.go:64] FLAG: --logging-format="text"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686940    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686945    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686950    2582 flags.go:64] FLAG: --manifest-url=""
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686954    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686961    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686965    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686971    2582 flags.go:64] FLAG: --max-pods="110"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686976    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686981    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686985    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686990    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686995    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 18:00:18.695083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.686999    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687004    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687017    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687021    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687026    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687031    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687036    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687064    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687070    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687075    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687080    2582 flags.go:64] FLAG: --port="10250"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687084    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687089    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03be9e41844ea49ed"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687094    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687103    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687108    2582 flags.go:64] FLAG: --register-node="true"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687113    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687118    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687123    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687128    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687132    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687138    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687143    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687148    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687152    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 18:00:18.695704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687157    2582 flags.go:64] FLAG: --runonce="false"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687161    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687166    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687171    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687176    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687181    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687185    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687190    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687194    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687201    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687206    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687210    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687215    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687220    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687224    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687229    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687237    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687241    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687245    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687251    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687255    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687260    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687265    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687270    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687275    2582 flags.go:64] FLAG: --v="2"
Sep 18 18:00:18.696715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687281    2582 flags.go:64] FLAG: --version="false"
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687286    2582 flags.go:64] FLAG: --vmodule=""
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687292    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.687298    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687493    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687502    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687507    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687512    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687517    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687521    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687525    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687529    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687534    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687538    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687542    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687546    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687552    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687557    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687561    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687565    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:18.697374 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687569    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687573    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687577    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687581    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687585    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687592    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687598    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687603    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687607    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687611    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687616    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687620    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687624    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687628    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687632    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687636    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687640    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687644    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687649    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687653    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:18.697868 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687657    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687661    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687665    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687669    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687673    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687678    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687682    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687686    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687692    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687696    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687700    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687704    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687708    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687712    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687716    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687720    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687723    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687728    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687732    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:18.698375 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687736    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687740    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687745    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687749    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687755    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687760    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687764    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687768    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687772    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687776    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687780    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687787    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687792    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687796    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687800    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687805    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687810    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687814    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687818    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687822    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:18.698826 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687827    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687832    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687837    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687840    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687844    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687849    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687853    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687857    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687861    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687865    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687869    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687873    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687877    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687881    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687885    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687889    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687895    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687900    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687904    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687908    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:18.699382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687912    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687917    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687921    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.687924    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.688744    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.697587    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.697602    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.697627    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.697633    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697687    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697692    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697695    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697698    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697701    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697704    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:18.699864 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697707    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697710    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697713    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697716    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697719    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697722    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697725    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697728    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697731    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697733    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697736    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697739    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697742    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697744    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697747    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697750    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697753    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697756    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697758    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697761    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:18.700269 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697763    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697767    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697771    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697774    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697777    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697780    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697783    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697786    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697789    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697791    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697794    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697797    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697799    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697802    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697804    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697808    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697811    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697814    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697817    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:18.700762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697820    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697822    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697825    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697827    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697830    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697832    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697835    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697838    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697840    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697843    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697846    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697848    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697851    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697853    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697856    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697858    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697862    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697864    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697867    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697869    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:18.701239 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697872    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697875    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697877    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697880    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697882    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697885    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697887    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697890    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697893    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697895    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697898    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697900    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697903    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697905    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697907    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697910    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697913    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697915    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697918    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697920    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:18.701714 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697923    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697926    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697928    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697930    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697933    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697935    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697938    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697940    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697943    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697945    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697948    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697951    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697955    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.697958    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:18.702248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.697962    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698115    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698121    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698124    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698127    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698130    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698133    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698136    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698138    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698141    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698144    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698146    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698149    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698152    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698154    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698156    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698159    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698161    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698164    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698167    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:18.702589 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698170    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698172    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698174    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698177    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698180    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698182    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698185    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698187    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698190    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698192    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698196    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698200    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698203    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698206    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698209    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698212    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698215    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698218    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698222    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:18.703099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698225    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698228    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698231    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698234    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698236    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698239    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698242    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698245    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698248    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698250    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698252    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698255    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698259    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698261    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698264    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698266    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698269    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698271    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698274    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698276    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:18.703583 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698279    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698281    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698283    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698286    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698288    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698291    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698293    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698296    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698298    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698301    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698303    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698306    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698308    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698311    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698313    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698316    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698320    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698323    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698326    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:18.704085 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698329    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698332    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698335    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698338    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698340    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698343    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698345    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698348    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698350    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698353    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698355    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698358    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698361    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698363    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698365    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698368    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698371    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698373    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698376    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698379    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:18.704580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698381    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:18.698384    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.698389    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.698494    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.700516    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.701539    2582 server.go:1019] "Starting client certificate rotation"
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.701642    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.701680    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:00:18.705114 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.704100    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:00:18.705382 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.705229    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:00:18.709365 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.709348    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 18 18:00:18.712113 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.712099    2582 log.go:25] "Validated CRI v1 image API"
Sep 18 18:00:18.712906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.712890    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 18:00:18.716082 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.716063    2582 fs.go:135] Filesystem UUIDs: map[61dc5482-756a-45e6-9faa-889d7a1749a3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9d70a8c1-e436-4070-a939-f61128f674bb:/dev/nvme0n1p3]
Sep 18 18:00:18.716151 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.716082    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 18:00:18.720888 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.720782    2582 manager.go:216] Machine: {Timestamp:2026-09-18 18:00:18.720318876 +0000 UTC m=+2.147531621 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090235 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20dac184d28d2a80990a2ad1bfea4f SystemUUID:ec20dac1-84d2-8d2a-8099-0a2ad1bfea4f BootID:7f4f356a-e538-4abf-85bb-e7dad418fc3a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c0:52:7c:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c0:52:7c:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:e5:bf:a2:8a:e1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 18:00:18.720888 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.720883    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 18:00:18.720997 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.720952    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 18:00:18.722624 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.722598    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 18:00:18.722748 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.722628    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-209.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 18:00:18.723856 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.723845    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 18:00:18.723891 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.723858    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 18:00:18.723891 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.723871    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:00:18.723891 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.723881    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:00:18.723976 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.723945    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 18:00:18.728309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728298    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:00:18.728420 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728412    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 18:00:18.728456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728444    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 18:00:18.728456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728454    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 18:00:18.728509 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728468    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 18:00:18.728509 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728477    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 18 18:00:18.728509 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.728486    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 18:00:18.732590 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.732562    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:00:18.732680 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.732643    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:00:18.736602 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.736579    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 18:00:18.736685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.736646    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:00:18.738720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.738706    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 18:00:18.738771 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.738737    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 18:00:18.739773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739756    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 18:00:18.739805 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739785    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 18:00:18.739805 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739794    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 18:00:18.739805 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739802    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739815    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739824    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739842    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739848    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739854    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739859    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739868    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 18:00:18.739895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739873    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 18:00:18.740117 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739899    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 18:00:18.740117 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.739907    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 18:00:18.741602 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.741579    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 18:00:18.741673 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.741578    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 18:00:18.743860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.743847    2582 server.go:1268] "Started kubelet"
Sep 18 18:00:18.743921 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.743888    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 18:00:18.744016 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.743966    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 18:00:18.744073 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.744042    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 18:00:18.744644 ip-10-0-135-209 systemd[1]: Started Kubernetes Kubelet.
Sep 18 18:00:18.745298 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.745271    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 18:00:18.745630 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.745560    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 18:00:18.748993 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.748970    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 18:00:18.752472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.752459    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 18:00:18.752544 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.752472    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 18:00:18.752825 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.752587    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:18.753069 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.753040    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 18:00:18.753245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.753224    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 18:00:18.753245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.753243    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 18:00:18.753405 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.753320    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 18:00:18.753405 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.753329    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 18:00:18.754172 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754157    2582 factory.go:153] Registering CRI-O factory
Sep 18 18:00:18.754248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754176    2582 factory.go:223] Registration of the crio container factory successfully
Sep 18 18:00:18.754248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754224    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 18:00:18.754248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754231    2582 factory.go:55] Registering systemd factory
Sep 18 18:00:18.754248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754236    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 18 18:00:18.754248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754247    2582 factory.go:103] Registering Raw factory
Sep 18 18:00:18.754428 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754256    2582 manager.go:1196] Started watching for new ooms in manager
Sep 18 18:00:18.754851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.754838    2582 manager.go:349] Starting recovery of all containers
Sep 18 18:00:18.777332 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.777312    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-209.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 18:00:18.777421 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.777383    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 18:00:18.777421 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.777281    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 18:00:18.783941 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.783795    2582 manager.go:354] Recovery completed
Sep 18 18:00:18.784039 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.782857    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-209.ec2.internal.18d67c294c693d40  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-209.ec2.internal,UID:ip-10-0-135-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-209.ec2.internal,},FirstTimestamp:2026-09-18 18:00:18.743819584 +0000 UTC m=+2.171032330,LastTimestamp:2026-09-18 18:00:18.743819584 +0000 UTC m=+2.171032330,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-209.ec2.internal,}"
Sep 18 18:00:18.787900 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.787888    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:18.788606 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.788589    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-grc5m"
Sep 18 18:00:18.790735 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.790721    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:18.790794 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.790747    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:18.790794 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.790758    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:18.791298 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791286    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:18.791368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791289    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 18:00:18.791368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791328    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 18:00:18.791368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791344    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:00:18.791953 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791938    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:18.792027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791962    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:18.792027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.791972    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:18.793562 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.793549    2582 policy_none.go:49] "None policy: Start"
Sep 18 18:00:18.793602 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.793569    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 18:00:18.793602 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.793582    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 18:00:18.795563 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.795553    2582 policy_none.go:47] "Start"
Sep 18 18:00:18.796791 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.796776    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-grc5m"
Sep 18 18:00:18.801525 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.801440    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-209.ec2.internal.18d67c294f35190f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-209.ec2.internal,UID:ip-10-0-135-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-209.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-209.ec2.internal,},FirstTimestamp:2026-09-18 18:00:18.790734095 +0000 UTC m=+2.217946839,LastTimestamp:2026-09-18 18:00:18.790734095 +0000 UTC m=+2.217946839,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-209.ec2.internal,}"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.831570    2582 manager.go:337] "Starting Device Plugin manager"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.831602    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.831615    2582 server.go:87] "Starting device plugin registration server"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.831845    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.831866    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.831961    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.832089    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.832111    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.832578    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 18:00:18.845152 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.832631    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:18.911937 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.911910    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 18:00:18.913225 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.913202    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 18:00:18.913311 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.913246    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 18:00:18.913311 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.913267    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 18:00:18.913311 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.913302    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 18:00:18.915620 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.915595    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:18.932412 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.932372    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:18.933182 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.933169    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:18.933265 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.933194    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:18.933265 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.933210    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:18.933265 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.933231    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-209.ec2.internal"
Sep 18 18:00:18.943878 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:18.943861    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-209.ec2.internal"
Sep 18 18:00:18.943932 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.943880    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-209.ec2.internal\": node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:18.977767 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:18.977742    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.014423 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.014375    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"]
Sep 18 18:00:19.014502 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.014460    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:19.015964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.015944    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:19.016083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.015976    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:19.016083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.015993    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:19.018158 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018144    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:19.018272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018257    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.018323 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018289    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:19.018898 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018883    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:19.018951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018913    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:19.018951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018925    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:19.019009 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018883    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:19.019009 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.018993    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:19.019009 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.019007    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:19.020977 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.020962    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.021040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.020988    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:19.022685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.021919    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:19.022685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.021943    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:19.022685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.021955    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:19.036593 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.036573    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-209.ec2.internal\" not found" node="ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.038704 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.038689    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-209.ec2.internal\" not found" node="ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.054800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.054779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3111df1f51d50ff2c58d32065c7c257b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal\" (UID: \"3111df1f51d50ff2c58d32065c7c257b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.054901 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.054814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3111df1f51d50ff2c58d32065c7c257b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal\" (UID: \"3111df1f51d50ff2c58d32065c7c257b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.054901 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.054839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cccdef02f8e26bf3341296b9df4a415f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-209.ec2.internal\" (UID: \"cccdef02f8e26bf3341296b9df4a415f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.077976 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.077957    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.155272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.155248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3111df1f51d50ff2c58d32065c7c257b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal\" (UID: \"3111df1f51d50ff2c58d32065c7c257b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.155373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.155278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3111df1f51d50ff2c58d32065c7c257b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal\" (UID: \"3111df1f51d50ff2c58d32065c7c257b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.155373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.155301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cccdef02f8e26bf3341296b9df4a415f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-209.ec2.internal\" (UID: \"cccdef02f8e26bf3341296b9df4a415f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.155373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.155349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3111df1f51d50ff2c58d32065c7c257b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal\" (UID: \"3111df1f51d50ff2c58d32065c7c257b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.155486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.155349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3111df1f51d50ff2c58d32065c7c257b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal\" (UID: \"3111df1f51d50ff2c58d32065c7c257b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.155486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.155356    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cccdef02f8e26bf3341296b9df4a415f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-209.ec2.internal\" (UID: \"cccdef02f8e26bf3341296b9df4a415f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.178378 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.178358    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.279205 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.279144    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.340349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.340317    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.343494 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.343470    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.380070 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.380034    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.480501 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.480475    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.581011 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.580937    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.681398 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.681371    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.701535 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.701514    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 18:00:19.701666 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.701650    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:00:19.782187 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:19.782165    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-209.ec2.internal\" not found"
Sep 18 18:00:19.801757 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.801731    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 17:55:18 +0000 UTC" deadline="2028-04-26 11:06:26.396871839 +0000 UTC"
Sep 18 18:00:19.801757 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.801754    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14057h6m6.59512105s"
Sep 18 18:00:19.848006 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.847955    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:19.849090 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.849074    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:19.849153 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.849104    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:19.849153 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.849114    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:19.858878 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.858862    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:19.952922 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.952884    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.962535 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.962517    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:00:19.964893 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.964881    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal"
Sep 18 18:00:19.977078 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:19.977037    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:00:20.155841 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.155758    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:20.254493 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.254469    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:20.434739 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:20.434673    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcccdef02f8e26bf3341296b9df4a415f.slice/crio-cc58047d6e278d6408112d8e15faf78e9069185c03e7c59a0294ee608f72fe1f WatchSource:0}: Error finding container cc58047d6e278d6408112d8e15faf78e9069185c03e7c59a0294ee608f72fe1f: Status 404 returned error can't find the container with id cc58047d6e278d6408112d8e15faf78e9069185c03e7c59a0294ee608f72fe1f
Sep 18 18:00:20.435852 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:20.435824    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3111df1f51d50ff2c58d32065c7c257b.slice/crio-d03f591eb7b1b0821f85a56e85f5248f574477d40f06c3a92eb212c1cb55cb51 WatchSource:0}: Error finding container d03f591eb7b1b0821f85a56e85f5248f574477d40f06c3a92eb212c1cb55cb51: Status 404 returned error can't find the container with id d03f591eb7b1b0821f85a56e85f5248f574477d40f06c3a92eb212c1cb55cb51
Sep 18 18:00:20.440600 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.440584    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:00:20.730281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.730213    2582 apiserver.go:52] "Watching apiserver"
Sep 18 18:00:20.735736 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.735719    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 18:00:20.736099 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.736078    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal","openshift-multus/multus-additional-cni-plugins-7sqh6","openshift-network-operator/iptables-alerter-6d27l","openshift-ovn-kubernetes/ovnkube-node-hbxhg","kube-system/global-pull-secret-syncer-8grrr","kube-system/konnectivity-agent-pdzvz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64","openshift-cluster-node-tuning-operator/tuned-c9krd","openshift-dns/node-resolver-trjfw","openshift-multus/multus-6d2zm","openshift-multus/network-metrics-daemon-d9bp5","openshift-network-diagnostics/network-check-target-xp6p5","kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal","openshift-image-registry/node-ca-6fz2m"]
Sep 18 18:00:20.739856 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.739840    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.741959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.741940    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.744156 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744131    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.744301 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744235    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 18:00:20.744595 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744574    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.744687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744606    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 18:00:20.744687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744603    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn946\""
Sep 18 18:00:20.744687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744642    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p5kz7\""
Sep 18 18:00:20.744687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.744585    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.746442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.746272    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.746442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.746332    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dpl9x\""
Sep 18 18:00:20.746442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.746337    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 18:00:20.746442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.746391    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.746826 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.746809    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.748973    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pb58m\""
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.749033    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.749311    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.749357    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.749374    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.749313    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.749914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.749648    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 18:00:20.751239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.751220    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.751324 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.751297    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:20.753219 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.753147    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 18:00:20.753893 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.753874    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.755813 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.755796    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.756023 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.756004    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 18:00:20.756202 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.756188    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.756283 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.756247    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7lg48\""
Sep 18 18:00:20.756387 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.756248    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.758155 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758140    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.758555 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758537    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 18:00:20.758626 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758539    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 18:00:20.758626 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758598    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 18:00:20.758778 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758761    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nj6s7\""
Sep 18 18:00:20.758832 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758795    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.758882 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.758863    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.761139 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.761120    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.761207 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.761149    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rm9dq\""
Sep 18 18:00:20.761207 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.761122    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.761316 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.761225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.763245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-lib-modules\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763257    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gkfj\" (UniqueName: \"kubernetes.io/projected/d3b49339-3e4b-4444-b400-a8a675199757-kube-api-access-8gkfj\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-env-overrides\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/97f840ed-745a-4adf-883d-6de9b99dfd41-ovn-node-metrics-cert\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b0445beb-a1ca-4b60-be4a-8f040edeab39-konnectivity-ca\") pod \"konnectivity-agent-pdzvz\" (UID: \"b0445beb-a1ca-4b60-be4a-8f040edeab39\") " pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763313    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763323    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9qxwv\""
Sep 18 18:00:20.763342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763316    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-kubernetes\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-var-lib-kubelet\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-systemd-units\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763433    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-slash\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763456    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-var-lib-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763469    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-log-socket\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.763521    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-run\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-systemd\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.763609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763580    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-cni-netd\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763611    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69zhv\" (UniqueName: \"kubernetes.io/projected/97f840ed-745a-4adf-883d-6de9b99dfd41-kube-api-access-69zhv\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6a1fbbd-d154-4857-9e94-f4a938082b6c-dbus\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-iptables-alerter-script\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gczv\" (UniqueName: \"kubernetes.io/projected/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-kube-api-access-2gczv\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/84010937-7373-4972-9b19-068e78b41043-etc-tuned\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763720    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-cni-bin\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763740    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-ovnkube-config\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763760    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-device-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763773    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-run-netns\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-ovn\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763849    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-host-slash\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysconfig\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-socket-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.764091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763885    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-sys-fs\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-etc-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b0445beb-a1ca-4b60-be4a-8f040edeab39-agent-certs\") pod \"konnectivity-agent-pdzvz\" (UID: \"b0445beb-a1ca-4b60-be4a-8f040edeab39\") " pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.763975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysctl-conf\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764003    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-systemd\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84010937-7373-4972-9b19-068e78b41043-tmp\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764032    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-registration-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764094    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-kubelet\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-modprobe-d\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-host\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k24jc\" (UniqueName: \"kubernetes.io/projected/84010937-7373-4972-9b19-068e78b41043-kube-api-access-k24jc\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-etc-selinux\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-node-log\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764224    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-run-ovn-kubernetes\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.764578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-ovnkube-script-lib\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.765014 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764255    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6a1fbbd-d154-4857-9e94-f4a938082b6c-kubelet-config\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.765014 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysctl-d\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.765014 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.764298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-sys\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.765851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.765836    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:20.765924 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.765890    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:20.768080 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.768065    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.770458 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.770441    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 18:00:20.770548 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.770478    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 18:00:20.770548 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.770523    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 18:00:20.770634 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.770600    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m8fs6\""
Sep 18 18:00:20.774016 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.774002    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:20.789139 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.789124    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b72h7"
Sep 18 18:00:20.797315 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.797300    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b72h7"
Sep 18 18:00:20.853738 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.853709    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 18:00:20.864472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-cni-bin\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.864589 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-ovnkube-config\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.864589 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864514    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc8b275-9b48-462a-a66f-659147ccc56d-tmp-dir\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.864589 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-run-netns\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.864589 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-netns\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.864589 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-cni-bin\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.864770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864607    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-run-netns\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.864770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-multus-certs\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.864770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysconfig\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.864770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864730    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-sys-fs\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.864770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-etc-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864772    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysconfig\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864784    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-sys-fs\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-etc-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk7lc\" (UniqueName: \"kubernetes.io/projected/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-kube-api-access-bk7lc\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864845    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b0445beb-a1ca-4b60-be4a-8f040edeab39-agent-certs\") pod \"konnectivity-agent-pdzvz\" (UID: \"b0445beb-a1ca-4b60-be4a-8f040edeab39\") " pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysctl-conf\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84010937-7373-4972-9b19-068e78b41043-tmp\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864965    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-kubelet\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.864987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-run-ovn-kubernetes\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865010    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-host\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k24jc\" (UniqueName: \"kubernetes.io/projected/84010937-7373-4972-9b19-068e78b41043-kube-api-access-k24jc\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-etc-selinux\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865116    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-run-ovn-kubernetes\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865125    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6a1fbbd-d154-4857-9e94-f4a938082b6c-kubelet-config\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865128    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysctl-conf\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-cnibin\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865167    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-kubelet\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-ovnkube-config\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-cni-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865225    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-cni-bin\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865270    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-kubelet\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865278    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6a1fbbd-d154-4857-9e94-f4a938082b6c-kubelet-config\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865310    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-host\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysctl-d\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.865557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-lib-modules\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865357    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-etc-selinux\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865418    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-sysctl-d\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865361    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-env-overrides\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-lib-modules\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-hostroot\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865529    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-etc-kubernetes\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b0445beb-a1ca-4b60-be4a-8f040edeab39-konnectivity-ca\") pod \"konnectivity-agent-pdzvz\" (UID: \"b0445beb-a1ca-4b60-be4a-8f040edeab39\") " pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-var-lib-kubelet\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865713    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-var-lib-kubelet\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865726    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-env-overrides\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-var-lib-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-var-lib-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69zhv\" (UniqueName: \"kubernetes.io/projected/97f840ed-745a-4adf-883d-6de9b99dfd41-kube-api-access-69zhv\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-iptables-alerter-script\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.866302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865918    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-openvswitch\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.865923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gczv\" (UniqueName: \"kubernetes.io/projected/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-kube-api-access-2gczv\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b0445beb-a1ca-4b60-be4a-8f040edeab39-konnectivity-ca\") pod \"konnectivity-agent-pdzvz\" (UID: \"b0445beb-a1ca-4b60-be4a-8f040edeab39\") " pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866066    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd8c8072-1646-495e-87c0-56da94590049-multus-daemon-config\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-run\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-cni-netd\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866139    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-run\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6a1fbbd-d154-4857-9e94-f4a938082b6c-dbus\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3dc8b275-9b48-462a-a66f-659147ccc56d-hosts-file\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-cni-netd\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/84010937-7373-4972-9b19-068e78b41043-etc-tuned\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd8c8072-1646-495e-87c0-56da94590049-cni-binary-copy\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866268    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6a1fbbd-d154-4857-9e94-f4a938082b6c-dbus\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-iptables-alerter-script\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866415    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-host\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-device-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-ovn\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.866971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866531    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866535    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-device-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-host-slash\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866575    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-ovn\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-system-cni-dir\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866600    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-host-slash\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866586    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866619    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866646    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-socket-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66295\" (UniqueName: \"kubernetes.io/projected/0497d646-848f-4d71-b54c-db65e05579ee-kube-api-access-66295\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-system-cni-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-socket-dir-parent\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-systemd\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-socket-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-registration-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866799    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-systemd\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.867773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-node-log\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-ovnkube-script-lib\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866880    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866885    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-node-log\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.866910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-cnibin\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.866954    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.867070    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:21.367031311 +0000 UTC m=+4.794244060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-modprobe-d\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl5qt\" (UniqueName: \"kubernetes.io/projected/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-kube-api-access-bl5qt\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-sys\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867191    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3b49339-3e4b-4444-b400-a8a675199757-registration-dir\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gkfj\" (UniqueName: \"kubernetes.io/projected/d3b49339-3e4b-4444-b400-a8a675199757-kube-api-access-8gkfj\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/97f840ed-745a-4adf-883d-6de9b99dfd41-ovn-node-metrics-cert\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867334    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-k8s-cni-cncf-io\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867351    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-cni-multus\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-conf-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.868464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867397    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867398    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/97f840ed-745a-4adf-883d-6de9b99dfd41-ovnkube-script-lib\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xbh4\" (UniqueName: \"kubernetes.io/projected/3dc8b275-9b48-462a-a66f-659147ccc56d-kube-api-access-5xbh4\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867506    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-modprobe-d\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867542    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-kubernetes\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-systemd-units\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-slash\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-log-socket\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-serviceca\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-systemd\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867745    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzlcz\" (UniqueName: \"kubernetes.io/projected/fd8c8072-1646-495e-87c0-56da94590049-kube-api-access-rzlcz\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867845    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-log-socket\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-sys\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867927    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84010937-7373-4972-9b19-068e78b41043-etc-kubernetes\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.867991    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-systemd-units\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.868030    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-host-slash\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.868094    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/97f840ed-745a-4adf-883d-6de9b99dfd41-run-systemd\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.869007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.868989    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/84010937-7373-4972-9b19-068e78b41043-etc-tuned\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.869650 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.869096    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84010937-7373-4972-9b19-068e78b41043-tmp\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.869650 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.869378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b0445beb-a1ca-4b60-be4a-8f040edeab39-agent-certs\") pod \"konnectivity-agent-pdzvz\" (UID: \"b0445beb-a1ca-4b60-be4a-8f040edeab39\") " pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:20.869650 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.869396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/97f840ed-745a-4adf-883d-6de9b99dfd41-ovn-node-metrics-cert\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.873367 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.873292    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k24jc\" (UniqueName: \"kubernetes.io/projected/84010937-7373-4972-9b19-068e78b41043-kube-api-access-k24jc\") pod \"tuned-c9krd\" (UID: \"84010937-7373-4972-9b19-068e78b41043\") " pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:20.873826 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.873808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gczv\" (UniqueName: \"kubernetes.io/projected/9a1e5a8c-4d32-45e1-b11e-f28273951b3c-kube-api-access-2gczv\") pod \"iptables-alerter-6d27l\" (UID: \"9a1e5a8c-4d32-45e1-b11e-f28273951b3c\") " pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:20.874206 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.874188    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69zhv\" (UniqueName: \"kubernetes.io/projected/97f840ed-745a-4adf-883d-6de9b99dfd41-kube-api-access-69zhv\") pod \"ovnkube-node-hbxhg\" (UID: \"97f840ed-745a-4adf-883d-6de9b99dfd41\") " pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:20.874871 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.874854    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gkfj\" (UniqueName: \"kubernetes.io/projected/d3b49339-3e4b-4444-b400-a8a675199757-kube-api-access-8gkfj\") pod \"aws-ebs-csi-driver-node-mzr64\" (UID: \"d3b49339-3e4b-4444-b400-a8a675199757\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:20.917552 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.917510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal" event={"ID":"3111df1f51d50ff2c58d32065c7c257b","Type":"ContainerStarted","Data":"d03f591eb7b1b0821f85a56e85f5248f574477d40f06c3a92eb212c1cb55cb51"}
Sep 18 18:00:20.918461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.918437    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal" event={"ID":"cccdef02f8e26bf3341296b9df4a415f","Type":"ContainerStarted","Data":"cc58047d6e278d6408112d8e15faf78e9069185c03e7c59a0294ee608f72fe1f"}
Sep 18 18:00:20.968515 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968484    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:20.968647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xbh4\" (UniqueName: \"kubernetes.io/projected/3dc8b275-9b48-462a-a66f-659147ccc56d-kube-api-access-5xbh4\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.968647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968551    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-serviceca\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.968647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzlcz\" (UniqueName: \"kubernetes.io/projected/fd8c8072-1646-495e-87c0-56da94590049-kube-api-access-rzlcz\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.968647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc8b275-9b48-462a-a66f-659147ccc56d-tmp-dir\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.968859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-netns\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.968859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968787    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-multus-certs\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.968859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.968859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968837    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc8b275-9b48-462a-a66f-659147ccc56d-tmp-dir\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.969075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968894    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-netns\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968943    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-multus-certs\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969192 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969168    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-serviceca\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.969259 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.968841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk7lc\" (UniqueName: \"kubernetes.io/projected/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-kube-api-access-bk7lc\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.969259 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969259 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-cnibin\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-cni-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-cni-bin\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-kubelet\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-hostroot\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-etc-kubernetes\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969369    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969387    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd8c8072-1646-495e-87c0-56da94590049-multus-daemon-config\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3dc8b275-9b48-462a-a66f-659147ccc56d-hosts-file\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969409    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd8c8072-1646-495e-87c0-56da94590049-cni-binary-copy\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-host\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-kubelet\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-system-cni-dir\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969480    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-cnibin\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969487    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66295\" (UniqueName: \"kubernetes.io/projected/0497d646-848f-4d71-b54c-db65e05579ee-kube-api-access-66295\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969516    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-cni-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-system-cni-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-cni-bin\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-socket-dir-parent\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.969578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969573    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-socket-dir-parent\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-cnibin\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969609    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-hostroot\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl5qt\" (UniqueName: \"kubernetes.io/projected/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-kube-api-access-bl5qt\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-k8s-cni-cncf-io\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-cni-multus\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969670    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-conf-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969716    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-multus-conf-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969743    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-etc-kubernetes\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.969806    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.969822    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-cnibin\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.969847    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:21.469833637 +0000 UTC m=+4.897046370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970297    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.970325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970321    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0497d646-848f-4d71-b54c-db65e05579ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.970860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970437    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.970860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970659    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-system-cni-dir\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-run-k8s-cni-cncf-io\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970730    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd8c8072-1646-495e-87c0-56da94590049-host-var-lib-cni-multus\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970770    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd8c8072-1646-495e-87c0-56da94590049-multus-daemon-config\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.970860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3dc8b275-9b48-462a-a66f-659147ccc56d-hosts-file\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.971025 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970912    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-host\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.971025 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.970951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0497d646-848f-4d71-b54c-db65e05579ee-system-cni-dir\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:20.971249 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.971231    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd8c8072-1646-495e-87c0-56da94590049-cni-binary-copy\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.976232 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.976202    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:20.976232 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.976227    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:20.976392 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.976241    2582 projected.go:196] Error preparing data for projected volume kube-api-access-qtq6m for pod openshift-network-diagnostics/network-check-target-xp6p5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:20.976392 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:20.976303    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m podName:439cdf3d-8f3f-43a1-bfe7-8609d7456576 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:21.476286479 +0000 UTC m=+4.903499232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qtq6m" (UniqueName: "kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m") pod "network-check-target-xp6p5" (UID: "439cdf3d-8f3f-43a1-bfe7-8609d7456576") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:20.977594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.977574    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzlcz\" (UniqueName: \"kubernetes.io/projected/fd8c8072-1646-495e-87c0-56da94590049-kube-api-access-rzlcz\") pod \"multus-6d2zm\" (UID: \"fd8c8072-1646-495e-87c0-56da94590049\") " pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:20.977689 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.977661    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl5qt\" (UniqueName: \"kubernetes.io/projected/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-kube-api-access-bl5qt\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:20.978815 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.978793    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xbh4\" (UniqueName: \"kubernetes.io/projected/3dc8b275-9b48-462a-a66f-659147ccc56d-kube-api-access-5xbh4\") pod \"node-resolver-trjfw\" (UID: \"3dc8b275-9b48-462a-a66f-659147ccc56d\") " pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:20.979002 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.978979    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk7lc\" (UniqueName: \"kubernetes.io/projected/dba2c0a8-d33d-4dc7-9b6b-0ec277193214-kube-api-access-bk7lc\") pod \"node-ca-6fz2m\" (UID: \"dba2c0a8-d33d-4dc7-9b6b-0ec277193214\") " pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:20.979253 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:20.979238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66295\" (UniqueName: \"kubernetes.io/projected/0497d646-848f-4d71-b54c-db65e05579ee-kube-api-access-66295\") pod \"multus-additional-cni-plugins-7sqh6\" (UID: \"0497d646-848f-4d71-b54c-db65e05579ee\") " pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:21.051720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.051477    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:21.056890 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.056866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c9krd"
Sep 18 18:00:21.063334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.063314    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64"
Sep 18 18:00:21.063548 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.063525    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0445beb_a1ca_4b60_be4a_8f040edeab39.slice/crio-646e9be7f0de77a64a7c057b6f836976f0cf34dec2fece93b662a6bfdbf2fa5e WatchSource:0}: Error finding container 646e9be7f0de77a64a7c057b6f836976f0cf34dec2fece93b662a6bfdbf2fa5e: Status 404 returned error can't find the container with id 646e9be7f0de77a64a7c057b6f836976f0cf34dec2fece93b662a6bfdbf2fa5e
Sep 18 18:00:21.066903 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.066877    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84010937_7373_4972_9b19_068e78b41043.slice/crio-762c1e12756735c645d303931b40ccd5b56d2d0053dbdeb9eb49769e62e34978 WatchSource:0}: Error finding container 762c1e12756735c645d303931b40ccd5b56d2d0053dbdeb9eb49769e62e34978: Status 404 returned error can't find the container with id 762c1e12756735c645d303931b40ccd5b56d2d0053dbdeb9eb49769e62e34978
Sep 18 18:00:21.068157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.068004    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:21.075470 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.075292    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6d27l"
Sep 18 18:00:21.076073 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.076034    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b49339_3e4b_4444_b400_a8a675199757.slice/crio-9e4fae077b0c55689a7260efede564397a161184cf0485a3018503d7088208fb WatchSource:0}: Error finding container 9e4fae077b0c55689a7260efede564397a161184cf0485a3018503d7088208fb: Status 404 returned error can't find the container with id 9e4fae077b0c55689a7260efede564397a161184cf0485a3018503d7088208fb
Sep 18 18:00:21.080794 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.080461    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7sqh6"
Sep 18 18:00:21.080979 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.080958    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f840ed_745a_4adf_883d_6de9b99dfd41.slice/crio-f515a2e8e2dbff7f4f32113b959b469448c242f6f0a32f19ff7a88e029fd4217 WatchSource:0}: Error finding container f515a2e8e2dbff7f4f32113b959b469448c242f6f0a32f19ff7a88e029fd4217: Status 404 returned error can't find the container with id f515a2e8e2dbff7f4f32113b959b469448c242f6f0a32f19ff7a88e029fd4217
Sep 18 18:00:21.085528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.085511    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-trjfw"
Sep 18 18:00:21.090348 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.090327    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6d2zm"
Sep 18 18:00:21.095235 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.095217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6fz2m"
Sep 18 18:00:21.096324 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.096294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0497d646_848f_4d71_b54c_db65e05579ee.slice/crio-9e63e249987f59e34c581cfc6d235785359289a65ba45b1d1e1be0fbcf3f5865 WatchSource:0}: Error finding container 9e63e249987f59e34c581cfc6d235785359289a65ba45b1d1e1be0fbcf3f5865: Status 404 returned error can't find the container with id 9e63e249987f59e34c581cfc6d235785359289a65ba45b1d1e1be0fbcf3f5865
Sep 18 18:00:21.099720 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.099460    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc8b275_9b48_462a_a66f_659147ccc56d.slice/crio-951d4fdc927fb690bba625d8dbacc45901669fe062a9008cc857f9b7b67c90ba WatchSource:0}: Error finding container 951d4fdc927fb690bba625d8dbacc45901669fe062a9008cc857f9b7b67c90ba: Status 404 returned error can't find the container with id 951d4fdc927fb690bba625d8dbacc45901669fe062a9008cc857f9b7b67c90ba
Sep 18 18:00:21.105248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.105226    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd8c8072_1646_495e_87c0_56da94590049.slice/crio-523a24de5179900d0a8ebce66ba73ca294ef87701c47a03527364136427bc341 WatchSource:0}: Error finding container 523a24de5179900d0a8ebce66ba73ca294ef87701c47a03527364136427bc341: Status 404 returned error can't find the container with id 523a24de5179900d0a8ebce66ba73ca294ef87701c47a03527364136427bc341
Sep 18 18:00:21.108676 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:21.108659    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba2c0a8_d33d_4dc7_9b6b_0ec277193214.slice/crio-ae32f24fda4d75df59936442f56e361bf072456fff366c8b7f4147524ea9d774 WatchSource:0}: Error finding container ae32f24fda4d75df59936442f56e361bf072456fff366c8b7f4147524ea9d774: Status 404 returned error can't find the container with id ae32f24fda4d75df59936442f56e361bf072456fff366c8b7f4147524ea9d774
Sep 18 18:00:21.372931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.372861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:21.373114 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.373025    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:21.373114 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.373111    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:22.37308345 +0000 UTC m=+5.800296205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:21.473863 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.473833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:21.474032 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.474011    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:21.474125 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.474113    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:22.474083657 +0000 UTC m=+5.901296404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:21.575224 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.575181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:21.575398 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.575332    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:21.575398 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.575358    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:21.575398 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.575370    2582 projected.go:196] Error preparing data for projected volume kube-api-access-qtq6m for pod openshift-network-diagnostics/network-check-target-xp6p5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:21.575549 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.575442    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m podName:439cdf3d-8f3f-43a1-bfe7-8609d7456576 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:22.575422773 +0000 UTC m=+6.002635524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtq6m" (UniqueName: "kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m") pod "network-check-target-xp6p5" (UID: "439cdf3d-8f3f-43a1-bfe7-8609d7456576") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:21.801749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.801660    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 17:55:20 +0000 UTC" deadline="2028-03-17 01:43:51.139564067 +0000 UTC"
Sep 18 18:00:21.801749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.801696    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13087h43m29.337873047s"
Sep 18 18:00:21.913497 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.913467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:21.913666 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.913585    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:21.914034 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.914013    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:21.914175 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:21.914151    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:21.922538 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.922508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6d2zm" event={"ID":"fd8c8072-1646-495e-87c0-56da94590049","Type":"ContainerStarted","Data":"523a24de5179900d0a8ebce66ba73ca294ef87701c47a03527364136427bc341"}
Sep 18 18:00:21.924272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.924245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-trjfw" event={"ID":"3dc8b275-9b48-462a-a66f-659147ccc56d","Type":"ContainerStarted","Data":"951d4fdc927fb690bba625d8dbacc45901669fe062a9008cc857f9b7b67c90ba"}
Sep 18 18:00:21.926042 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.926016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerStarted","Data":"9e63e249987f59e34c581cfc6d235785359289a65ba45b1d1e1be0fbcf3f5865"}
Sep 18 18:00:21.927529 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.927502    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6d27l" event={"ID":"9a1e5a8c-4d32-45e1-b11e-f28273951b3c","Type":"ContainerStarted","Data":"df6275f2884cce3ddc03b2b37f9f68b68eb1646636f1a9fe2a349a7793009e60"}
Sep 18 18:00:21.930409 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.930374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64" event={"ID":"d3b49339-3e4b-4444-b400-a8a675199757","Type":"ContainerStarted","Data":"9e4fae077b0c55689a7260efede564397a161184cf0485a3018503d7088208fb"}
Sep 18 18:00:21.931805 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.931785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pdzvz" event={"ID":"b0445beb-a1ca-4b60-be4a-8f040edeab39","Type":"ContainerStarted","Data":"646e9be7f0de77a64a7c057b6f836976f0cf34dec2fece93b662a6bfdbf2fa5e"}
Sep 18 18:00:21.933960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.933935    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6fz2m" event={"ID":"dba2c0a8-d33d-4dc7-9b6b-0ec277193214","Type":"ContainerStarted","Data":"ae32f24fda4d75df59936442f56e361bf072456fff366c8b7f4147524ea9d774"}
Sep 18 18:00:21.935707 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.935681    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"f515a2e8e2dbff7f4f32113b959b469448c242f6f0a32f19ff7a88e029fd4217"}
Sep 18 18:00:21.937079 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:21.937038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c9krd" event={"ID":"84010937-7373-4972-9b19-068e78b41043","Type":"ContainerStarted","Data":"762c1e12756735c645d303931b40ccd5b56d2d0053dbdeb9eb49769e62e34978"}
Sep 18 18:00:22.381569 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.381505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:22.381750 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.381701    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:22.381823 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.381762    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:24.381744254 +0000 UTC m=+7.808956986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:22.482520 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.482294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:22.482668 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.482522    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:22.482668 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.482580    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:24.482564252 +0000 UTC m=+7.909776998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:22.584608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.584574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:22.584802 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.584784    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:22.584889 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.584810    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:22.584889 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.584821    2582 projected.go:196] Error preparing data for projected volume kube-api-access-qtq6m for pod openshift-network-diagnostics/network-check-target-xp6p5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:22.584889 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.584880    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m podName:439cdf3d-8f3f-43a1-bfe7-8609d7456576 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:24.584861648 +0000 UTC m=+8.012074394 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtq6m" (UniqueName: "kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m") pod "network-check-target-xp6p5" (UID: "439cdf3d-8f3f-43a1-bfe7-8609d7456576") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:22.802865 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.802666    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 17:55:20 +0000 UTC" deadline="2028-03-16 06:56:28.192338351 +0000 UTC"
Sep 18 18:00:22.802865 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.802708    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13068h56m5.38963502s"
Sep 18 18:00:22.914235 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.914204    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:22.914453 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:22.914373    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:22.960959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:22.960922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal" event={"ID":"cccdef02f8e26bf3341296b9df4a415f","Type":"ContainerStarted","Data":"be7d979c606be0c5a6646efa1407220766866892ad9eb44dcd756e0c52163f2b"}
Sep 18 18:00:23.914328 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:23.914292    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:23.914763 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:23.914421    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:23.914900 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:23.914880    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:23.915021 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:23.915001    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:24.397819 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:24.397737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:24.397973 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.397884    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:24.397973 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.397941    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:28.397924514 +0000 UTC m=+11.825137252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:24.499098 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:24.499044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:24.499272 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.499212    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:24.499331 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.499273    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:28.499255432 +0000 UTC m=+11.926468178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:24.599865 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:24.599821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:24.600090 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.600018    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:24.600090 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.600042    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:24.600090 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.600070    2582 projected.go:196] Error preparing data for projected volume kube-api-access-qtq6m for pod openshift-network-diagnostics/network-check-target-xp6p5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:24.600239 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.600129    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m podName:439cdf3d-8f3f-43a1-bfe7-8609d7456576 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:28.600110024 +0000 UTC m=+12.027322764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtq6m" (UniqueName: "kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m") pod "network-check-target-xp6p5" (UID: "439cdf3d-8f3f-43a1-bfe7-8609d7456576") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:24.915848 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:24.915806    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:24.916348 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:24.915938    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:25.914275 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:25.914247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:25.914441 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:25.914346    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:25.914793 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:25.914630    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:25.914793 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:25.914739    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:26.914287 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:26.914243    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:26.914825 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:26.914381    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:27.913509 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:27.913477    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:27.913697 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:27.913490    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:27.913697 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:27.913583    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:27.913806 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:27.913698    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:28.429857 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:28.429825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:28.430250 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.429968    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:28.430250 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.430024    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:36.430007557 +0000 UTC m=+19.857220289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:28.530864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:28.530829    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:28.531066 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.531013    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:28.531207 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.531111    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:36.531090226 +0000 UTC m=+19.958302960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:28.632247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:28.632143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:28.632424 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.632306    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:28.632424 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.632326    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:28.632424 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.632336    2582 projected.go:196] Error preparing data for projected volume kube-api-access-qtq6m for pod openshift-network-diagnostics/network-check-target-xp6p5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:28.632424 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.632387    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m podName:439cdf3d-8f3f-43a1-bfe7-8609d7456576 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:36.632374427 +0000 UTC m=+20.059587159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtq6m" (UniqueName: "kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m") pod "network-check-target-xp6p5" (UID: "439cdf3d-8f3f-43a1-bfe7-8609d7456576") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:28.917978 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:28.917745    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:28.918167 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:28.918032    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:29.913705 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:29.913672    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:29.914102 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:29.913811    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:29.914102 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:29.913850    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:29.914102 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:29.913944    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:30.916153 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:30.915602    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:30.916153 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:30.915729    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:31.914286 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:31.914250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:31.914459 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:31.914369    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:31.914669 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:31.914251    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:31.914754 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:31.914739    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:32.913833 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:32.913801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:32.914289 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:32.913936    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:33.913934 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:33.913899    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:33.914422 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:33.913901    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:33.914422 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:33.914159    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:33.914422 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:33.914008    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:34.914157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:34.914112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:34.914569 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:34.914241    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:35.914232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:35.914193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:35.914721 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:35.914196    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:35.914721 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:35.914322    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:35.914721 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:35.914431    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:36.487841 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:36.487799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:36.488010 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.487928    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:36.488010 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.487997    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:52.48798065 +0000 UTC m=+35.915193401 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:36.588597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:36.588563    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:36.588762 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.588702    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:36.588810 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.588765    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:00:52.588745704 +0000 UTC m=+36.015958453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:36.689102 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:36.689064    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:36.689275 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.689192    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:36.689275 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.689215    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:36.689275 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.689228    2582 projected.go:196] Error preparing data for projected volume kube-api-access-qtq6m for pod openshift-network-diagnostics/network-check-target-xp6p5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:36.689405 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.689292    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m podName:439cdf3d-8f3f-43a1-bfe7-8609d7456576 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:52.68927474 +0000 UTC m=+36.116487472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtq6m" (UniqueName: "kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m") pod "network-check-target-xp6p5" (UID: "439cdf3d-8f3f-43a1-bfe7-8609d7456576") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:36.914186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:36.914154    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:36.914380 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:36.914286    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:37.914101 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:37.914068    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:37.914288 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:37.914165    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:37.914288 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:37.914225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:37.914401 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:37.914329    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:38.915326 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:38.915293    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:38.915725 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:38.915410    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:39.913745 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:39.913718    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:39.913879 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:39.913810    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:39.914238 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:39.914220    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:39.914347 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:39.914305    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:40.009042 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.008897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"316bc043c52f7353fe1896918379910fa8bcf2db8a072afa128aa8badee3a0c6"}
Sep 18 18:00:40.009042 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.008965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"aef00d26132a407badb4683877cde465ea71028600df1347348fb1de302358c6"}
Sep 18 18:00:40.009042 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.008995    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"17272e3e8fabb882440c79d62c374bef300e07dfbd009c611f8c61b335c105cb"}
Sep 18 18:00:40.011971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.011302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c9krd" event={"ID":"84010937-7373-4972-9b19-068e78b41043","Type":"ContainerStarted","Data":"ef9ee8e9093c8cf653e3f1ccae8b3703b157d49a404a8ac9fb1c61b90e8e6948"}
Sep 18 18:00:40.013470 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.013433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6d2zm" event={"ID":"fd8c8072-1646-495e-87c0-56da94590049","Type":"ContainerStarted","Data":"bcc4d5a5049359ead478d822434bc672bbfb56cd7fd9c674f821ce0873ea353d"}
Sep 18 18:00:40.034962 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.034806    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c9krd" podStartSLOduration=3.421840207 podStartE2EDuration="22.034787165s" totalImagesPullingTime="18.612946958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:40.034383124 +0000 UTC m=+23.461595879" watchObservedRunningTime="2026-09-18 18:00:40.034787165 +0000 UTC m=+23.461999912"
Sep 18 18:00:40.035373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.035334    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-209.ec2.internal" podStartSLOduration=21.035318974 podStartE2EDuration="21.035318974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:22.982003246 +0000 UTC m=+6.409216001" watchObservedRunningTime="2026-09-18 18:00:40.035318974 +0000 UTC m=+23.462531729"
Sep 18 18:00:40.053670 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.053561    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6d2zm" podStartSLOduration=2.41665237 podStartE2EDuration="21.053547003s" totalImagesPullingTime="18.636894633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:40.053073038 +0000 UTC m=+23.480285795" watchObservedRunningTime="2026-09-18 18:00:40.053547003 +0000 UTC m=+23.480759756"
Sep 18 18:00:40.913997 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:40.913967    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:40.914168 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:40.914093    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:41.016861 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.016831    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64" event={"ID":"d3b49339-3e4b-4444-b400-a8a675199757","Type":"ContainerStarted","Data":"06767cd4bfd47909589c7fffe0801dab8bf1981a796766ea41daa2c51c3f6f2e"}
Sep 18 18:00:41.018279 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.018257    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pdzvz" event={"ID":"b0445beb-a1ca-4b60-be4a-8f040edeab39","Type":"ContainerStarted","Data":"17ac7c15fdffef70eeceaf1c15f332d103cb67a87664b24cb1428785027998d7"}
Sep 18 18:00:41.019719 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.019701    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6fz2m" event={"ID":"dba2c0a8-d33d-4dc7-9b6b-0ec277193214","Type":"ContainerStarted","Data":"6c3ad9ff8cbf29a7cd8a9b187e42581509cd593d53dc01edd91b4b2399a20bf7"}
Sep 18 18:00:41.022995 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.022971    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"c0ea6356e3e49ea99da59e598ec6a58a61f80322877f8f4c8b8fced6e0776a2f"}
Sep 18 18:00:41.023237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.023030    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"637b039cf3a71a6ff73fddaf2a8179100e70f36f2328495cc03fd15f5a5451c1"}
Sep 18 18:00:41.023237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.023081    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"3d0dab7f8470c044d63182e90bc7f44c0fbcf244fbe303bf5d7da10a6405f2c5"}
Sep 18 18:00:41.024587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.024564    2582 generic.go:356] "Generic (PLEG): container finished" podID="3111df1f51d50ff2c58d32065c7c257b" containerID="3831434448e90c9b3e24ce9216d6e795bc675bbf1d5b4aa36dc33c4a4fe90f93" exitCode=0
Sep 18 18:00:41.024669 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.024639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal" event={"ID":"3111df1f51d50ff2c58d32065c7c257b","Type":"ContainerDied","Data":"3831434448e90c9b3e24ce9216d6e795bc675bbf1d5b4aa36dc33c4a4fe90f93"}
Sep 18 18:00:41.025951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.025922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-trjfw" event={"ID":"3dc8b275-9b48-462a-a66f-659147ccc56d","Type":"ContainerStarted","Data":"754a8f60242479271a9dffb8e4c7ee06a3a61f81c1b4d50196f1b745a64f9b4c"}
Sep 18 18:00:41.027449 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.027427    2582 generic.go:356] "Generic (PLEG): container finished" podID="0497d646-848f-4d71-b54c-db65e05579ee" containerID="6eaf1d150752d5552c76e1158494a84dcea33fd48835e721b42604905296647e" exitCode=0
Sep 18 18:00:41.027533 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.027479    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerDied","Data":"6eaf1d150752d5552c76e1158494a84dcea33fd48835e721b42604905296647e"}
Sep 18 18:00:41.029215 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.028985    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6d27l" event={"ID":"9a1e5a8c-4d32-45e1-b11e-f28273951b3c","Type":"ContainerStarted","Data":"f5d95b67ea1499ecc9b936a057d1c0c14751a960bcbf435dbf9ca95c611960d1"}
Sep 18 18:00:41.034087 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.034028    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pdzvz" podStartSLOduration=4.420412078 podStartE2EDuration="23.034017914s" totalImagesPullingTime="18.613605836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:41.033904807 +0000 UTC m=+24.461117560" watchObservedRunningTime="2026-09-18 18:00:41.034017914 +0000 UTC m=+24.461230668"
Sep 18 18:00:41.048359 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.048325    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6fz2m" podStartSLOduration=3.608035208 podStartE2EDuration="22.048311914s" totalImagesPullingTime="18.440276706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:41.047911992 +0000 UTC m=+24.475124746" watchObservedRunningTime="2026-09-18 18:00:41.048311914 +0000 UTC m=+24.475524684"
Sep 18 18:00:41.081911 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.081874    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6d27l" podStartSLOduration=7.714741159 podStartE2EDuration="22.081863201s" totalImagesPullingTime="14.367122042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:41.081750934 +0000 UTC m=+24.508963690" watchObservedRunningTime="2026-09-18 18:00:41.081863201 +0000 UTC m=+24.509076174"
Sep 18 18:00:41.119076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.118624    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-trjfw" podStartSLOduration=3.868447566 podStartE2EDuration="22.118606788s" totalImagesPullingTime="18.250159222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:41.11833868 +0000 UTC m=+24.545551455" watchObservedRunningTime="2026-09-18 18:00:41.118606788 +0000 UTC m=+24.545819544"
Sep 18 18:00:41.615690 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.615666    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 18:00:41.850101 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.849955    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T18:00:41.615686817Z","UUID":"516c0e53-85db-4ead-80d6-ccd563a275a8","Handler":null,"Name":"","Endpoint":""}
Sep 18 18:00:41.851365 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.851343    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 18:00:41.851506 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.851374    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 18:00:41.914194 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.914170    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:41.914347 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:41.914172    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:41.914347 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:41.914284    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:41.914489 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:41.914348    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:42.033503 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:42.033471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal" event={"ID":"3111df1f51d50ff2c58d32065c7c257b","Type":"ContainerStarted","Data":"d618f8d2668448568ca25e02d6893c56a7f5ac7fa5d973b60243adc18f54648c"}
Sep 18 18:00:42.035544 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:42.035507    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64" event={"ID":"d3b49339-3e4b-4444-b400-a8a675199757","Type":"ContainerStarted","Data":"d6da542113343dc917754ad1fd021c849e5b9afa8d546bea2931f38c53413521"}
Sep 18 18:00:42.049386 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:42.049349    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-209.ec2.internal" podStartSLOduration=23.049336389 podStartE2EDuration="23.049336389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:42.049130053 +0000 UTC m=+25.476342806" watchObservedRunningTime="2026-09-18 18:00:42.049336389 +0000 UTC m=+25.476549142"
Sep 18 18:00:42.913985 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:42.913908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:42.914198 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:42.914044    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:43.040324 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:43.040290    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64" event={"ID":"d3b49339-3e4b-4444-b400-a8a675199757","Type":"ContainerStarted","Data":"364c853d804b417f0a3cc9acf22eb66faf9bfb3a30055a5cb6269f33b413bb62"}
Sep 18 18:00:43.044665 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:43.044636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"273a2000cd8c2ce6776765b4bb5ebcf867de26adfc0639a0032ee0b6c40e5e87"}
Sep 18 18:00:43.057796 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:43.057750    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mzr64" podStartSLOduration=3.2902419849999998 podStartE2EDuration="24.057732983s" totalImagesPullingTime="20.767490998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:43.057640187 +0000 UTC m=+26.484852941" watchObservedRunningTime="2026-09-18 18:00:43.057732983 +0000 UTC m=+26.484945737"
Sep 18 18:00:43.913879 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:43.913850    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:43.914072 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:43.913850    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:43.914072 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:43.913965    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:43.914072 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:43.914025    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:44.914036 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:44.913957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:44.914439 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:44.914113    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:45.563239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:45.563194    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:45.564263 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:45.564246    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:45.914321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:45.914237    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:45.914321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:45.914223    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:45.914808 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:45.914373    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:45.914808 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:45.914506    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:46.914346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:46.914168    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:46.914944 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:46.914419    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:47.054628 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.054554    2582 generic.go:356] "Generic (PLEG): container finished" podID="0497d646-848f-4d71-b54c-db65e05579ee" containerID="0692e9c4edd177ef4878ba85105174c2b5310f94e125669502df9fac0aa69881" exitCode=0
Sep 18 18:00:47.054765 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.054645    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerDied","Data":"0692e9c4edd177ef4878ba85105174c2b5310f94e125669502df9fac0aa69881"}
Sep 18 18:00:47.058573 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.058551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" event={"ID":"97f840ed-745a-4adf-883d-6de9b99dfd41","Type":"ContainerStarted","Data":"f2e6b371f9cf42d69fb977eddfdac92e92bd217f11e63ea97cd600525847cc0e"}
Sep 18 18:00:47.058882 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.058860    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:47.074146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.074123    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:47.120449 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.120412    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg" podStartSLOduration=9.520065162 podStartE2EDuration="28.120398019s" totalImagesPullingTime="18.600332857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:47.119892397 +0000 UTC m=+30.547105175" watchObservedRunningTime="2026-09-18 18:00:47.120398019 +0000 UTC m=+30.547610773"
Sep 18 18:00:47.914136 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.914109    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:47.914291 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:47.914116    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:47.914291 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:47.914229    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:47.914291 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:47.914283    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:48.061890 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.061861    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:48.062043 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.061983    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:48.078175 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.078150    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:00:48.194683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.194655    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xp6p5"]
Sep 18 18:00:48.194832 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.194781    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:48.194924 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:48.194892    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:48.198806 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.198779    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8grrr"]
Sep 18 18:00:48.198908 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.198893    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:48.199021 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:48.198998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:48.199609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.199585    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d9bp5"]
Sep 18 18:00:48.199760 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:48.199693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:48.199818 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:48.199791    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:49.065643 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:49.065469    2582 generic.go:356] "Generic (PLEG): container finished" podID="0497d646-848f-4d71-b54c-db65e05579ee" containerID="2a76ec6a06bfe0bbf8ea933888d04004ff0b2a7f5ee512dbe2b50340ad113a07" exitCode=0
Sep 18 18:00:49.066022 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:49.065543    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerDied","Data":"2a76ec6a06bfe0bbf8ea933888d04004ff0b2a7f5ee512dbe2b50340ad113a07"}
Sep 18 18:00:49.914379 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:49.914348    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:49.914379 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:49.914383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:49.914575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:49.914459    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:49.914575 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:49.914463    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:49.914575 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:49.914524    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:49.914672 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:49.914588    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:50.632449 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:50.632378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:50.632821 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:50.632497    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 18:00:50.632889 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:50.632872    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pdzvz"
Sep 18 18:00:51.072744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:51.072716    2582 generic.go:356] "Generic (PLEG): container finished" podID="0497d646-848f-4d71-b54c-db65e05579ee" containerID="8b01a66c64a3cd3fbded339ba93c66e96072d2d96087fdebd7bdf596de0fd919" exitCode=0
Sep 18 18:00:51.072854 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:51.072792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerDied","Data":"8b01a66c64a3cd3fbded339ba93c66e96072d2d96087fdebd7bdf596de0fd919"}
Sep 18 18:00:51.914096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:51.914033    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:51.914096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:51.914074    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:51.914687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:51.914150    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:51.914687 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:51.914165    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp6p5" podUID="439cdf3d-8f3f-43a1-bfe7-8609d7456576"
Sep 18 18:00:51.914687 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:51.914248    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9bp5" podUID="d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c"
Sep 18 18:00:51.914687 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:51.914304    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8grrr" podUID="d6a1fbbd-d154-4857-9e94-f4a938082b6c"
Sep 18 18:00:52.413129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.413099    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-209.ec2.internal" event="NodeReady"
Sep 18 18:00:52.413307 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.413226    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 18:00:52.454773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.454741    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"]
Sep 18 18:00:52.474286 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.474260    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76756c9684-jxrnf"]
Sep 18 18:00:52.474440 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.474396    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.480539 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.479983    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.480539 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.479970    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 18 18:00:52.480539 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.480095    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.481119 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.481085    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 18 18:00:52.486806 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.486782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mgzll\""
Sep 18 18:00:52.490303 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.490276    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-tm2tl"]
Sep 18 18:00:52.490461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.490443    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.493849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.493824    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 18:00:52.495018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.494530    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 18:00:52.495018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.494606    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qzvvm\""
Sep 18 18:00:52.495018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.494766    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 18:00:52.501419 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.501395    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 18:00:52.502836 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.502813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:52.502946 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.502921    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:52.503012 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.502986    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret podName:d6a1fbbd-d154-4857-9e94-f4a938082b6c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.502967743 +0000 UTC m=+67.930180495 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret") pod "global-pull-secret-syncer-8grrr" (UID: "d6a1fbbd-d154-4857-9e94-f4a938082b6c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:52.505657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.505638    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"]
Sep 18 18:00:52.506602 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.506584    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.509606 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.509579    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-h8r5z\""
Sep 18 18:00:52.509704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.509678    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 18 18:00:52.510240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.510221    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.510443 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.510425    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.510525 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.510449    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 18 18:00:52.518024 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.517999    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 18 18:00:52.523073 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.523032    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-w8lzm"]
Sep 18 18:00:52.523172 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.523143    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:52.525455 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.525436    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.525554 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.525493    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vzjmc\""
Sep 18 18:00:52.525554 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.525503    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.526984 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.526965    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 18 18:00:52.545306 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.545283    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"]
Sep 18 18:00:52.545521 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.545413    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.548512 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.548491    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 18:00:52.548616 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.548514    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.548616 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.548586    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.549104 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.549079    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 18:00:52.549245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.549224    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-26666\""
Sep 18 18:00:52.553749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.553732    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 18:00:52.569357 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.569339    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"]
Sep 18 18:00:52.569475 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.569459    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.572309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.572289    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.572434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.572415    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 18 18:00:52.572626 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.572611    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.572921 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.572686    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 18 18:00:52.573152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.572747    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pwvsf\""
Sep 18 18:00:52.582361 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.582333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pdzvz_b0445beb-a1ca-4b60-be4a-8f040edeab39/konnectivity-agent/0.log" limit=1024
Sep 18 18:00:52.593886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.593867    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"]
Sep 18 18:00:52.594018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.593996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"
Sep 18 18:00:52.596290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.596270    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.596395 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.596299    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8btzp\""
Sep 18 18:00:52.596621 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.596601    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.603290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603150    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:52.603290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603188    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-certificates\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b17ce39-8f2d-4496-94d3-b62b3c984857-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.603290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj4p9\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-kube-api-access-jj4p9\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b17ce39-8f2d-4496-94d3-b62b3c984857-tmp\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nt78\" (UniqueName: \"kubernetes.io/projected/4b17ce39-8f2d-4496-94d3-b62b3c984857-kube-api-access-5nt78\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.603333    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603347    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38dfae07-7770-480a-941b-473232de5af7-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.603401    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs podName:d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.603383965 +0000 UTC m=+68.030596715 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs") pod "network-metrics-daemon-d9bp5" (UID: "d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603471    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b17ce39-8f2d-4496-94d3-b62b3c984857-serving-cert\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603572    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38dfae07-7770-480a-941b-473232de5af7-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.603609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb5vn\" (UniqueName: \"kubernetes.io/projected/2c502493-7814-4874-bac9-96021e28557d-kube-api-access-pb5vn\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603620    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-image-registry-private-configuration\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603636    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3a3834a-cc44-470d-8af5-665d4a6e5463-ca-trust-extracted\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmxm\" (UniqueName: \"kubernetes.io/projected/38dfae07-7770-480a-941b-473232de5af7-kube-api-access-cbmxm\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-trusted-ca\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-bound-sa-token\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4b17ce39-8f2d-4496-94d3-b62b3c984857-snapshots\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-installation-pull-secrets\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.603944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.603929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b17ce39-8f2d-4496-94d3-b62b3c984857-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.614213 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.614193    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-598c64547d-8cwhs"]
Sep 18 18:00:52.614321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.614307    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.617122 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.617100    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 18 18:00:52.617504 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.617486    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.617573 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.617535    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.617774 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.617754    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kk545\""
Sep 18 18:00:52.617851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.617824    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 18 18:00:52.630146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.630129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"]
Sep 18 18:00:52.630240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.630160    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"]
Sep 18 18:00:52.630240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.630184    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"]
Sep 18 18:00:52.630240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.630204    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-tm2tl"]
Sep 18 18:00:52.630240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.630225    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"]
Sep 18 18:00:52.630240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.630234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.632659 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.632641    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 18 18:00:52.632739 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.632651    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.632739 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.632685    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.633240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.633224    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 18 18:00:52.633240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.633231    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 18 18:00:52.633392 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.633224    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 18 18:00:52.633392 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.633252    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4rqgg\""
Sep 18 18:00:52.640194 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640170    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76756c9684-jxrnf"]
Sep 18 18:00:52.640272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640209    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-w8lzm"]
Sep 18 18:00:52.640272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640228    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-598c64547d-8cwhs"]
Sep 18 18:00:52.640272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640246    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"]
Sep 18 18:00:52.640272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"
Sep 18 18:00:52.640272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640263    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"]
Sep 18 18:00:52.640499 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640309    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"]
Sep 18 18:00:52.640499 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.640336    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bhppq"]
Sep 18 18:00:52.642561 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.642543    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.642632 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.642543    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.643744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.642945    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t5dwn\""
Sep 18 18:00:52.657881 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.657862    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vwp4c"]
Sep 18 18:00:52.657971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.657956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.660154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.660129    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 18:00:52.660154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.660149    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wnjwf\""
Sep 18 18:00:52.660324 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.660197    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 18:00:52.660324 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.660200    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 18:00:52.669826 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.669808    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bhppq"]
Sep 18 18:00:52.669906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.669838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vwp4c"]
Sep 18 18:00:52.669906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.669901    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.672202 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.672183    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 18:00:52.672285 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.672211    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 18:00:52.672335 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.672319    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhsq2\""
Sep 18 18:00:52.686168 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.686146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pdzvz_b0445beb-a1ca-4b60-be4a-8f040edeab39/konnectivity-agent/0.log" limit=1024
Sep 18 18:00:52.704971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.704950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nt78\" (UniqueName: \"kubernetes.io/projected/4b17ce39-8f2d-4496-94d3-b62b3c984857-kube-api-access-5nt78\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.705084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.704972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b17ce39-8f2d-4496-94d3-b62b3c984857-tmp\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.705084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.704999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a3558a4-49fc-4e96-986f-c1a7821da11a-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.705084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b17ce39-8f2d-4496-94d3-b62b3c984857-serving-cert\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.705084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c19150e6-187a-4e99-907e-b584d8b3aa55-config\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.705292 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:52.705292 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb5vn\" (UniqueName: \"kubernetes.io/projected/2c502493-7814-4874-bac9-96021e28557d-kube-api-access-pb5vn\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:52.705292 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4qqn\" (UniqueName: \"kubernetes.io/projected/f85d3569-b8be-47de-894a-7f2141464023-kube-api-access-t4qqn\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.705292 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.705208    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:00:52.705292 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/6a3558a4-49fc-4e96-986f-c1a7821da11a-kube-api-access-2jl5w\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.705292 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.705271    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls podName:2c502493-7814-4874-bac9-96021e28557d nodeName:}" failed. No retries permitted until 2026-09-18 18:00:53.20525251 +0000 UTC m=+36.632465249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gp99n" (UID: "2c502493-7814-4874-bac9-96021e28557d") : secret "samples-operator-tls" not found
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c19150e6-187a-4e99-907e-b584d8b3aa55-trusted-ca\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3a3834a-cc44-470d-8af5-665d4a6e5463-ca-trust-extracted\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpwg\" (UniqueName: \"kubernetes.io/projected/65231b2f-8527-44cc-9bc1-6629067f9311-kube-api-access-6mpwg\") pod \"volume-data-source-validator-5c4db8bdf9-ddlj2\" (UID: \"65231b2f-8527-44cc-9bc1-6629067f9311\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-trusted-ca\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705408    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b17ce39-8f2d-4496-94d3-b62b3c984857-tmp\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705456    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-default-certificate\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-stats-auth\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-installation-pull-secrets\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.705537    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.705555    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76756c9684-jxrnf: secret "image-registry-tls" not found
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705557    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmxm\" (UniqueName: \"kubernetes.io/projected/38dfae07-7770-480a-941b-473232de5af7-kube-api-access-cbmxm\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj4p9\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-kube-api-access-jj4p9\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.705610    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls podName:c3a3834a-cc44-470d-8af5-665d4a6e5463 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:53.205593048 +0000 UTC m=+36.632805802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls") pod "image-registry-76756c9684-jxrnf" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463") : secret "image-registry-tls" not found
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38dfae07-7770-480a-941b-473232de5af7-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.705709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkg8x\" (UniqueName: \"kubernetes.io/projected/ca87bb38-f503-4312-8ea0-ea8941f1ea67-kube-api-access-xkg8x\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38dfae07-7770-480a-941b-473232de5af7-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705717    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3a3834a-cc44-470d-8af5-665d4a6e5463-ca-trust-extracted\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-image-registry-private-configuration\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c19150e6-187a-4e99-907e-b584d8b3aa55-serving-cert\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705854    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f85d3569-b8be-47de-894a-7f2141464023-config\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-bound-sa-token\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4b17ce39-8f2d-4496-94d3-b62b3c984857-snapshots\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705968    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjzjr\" (UniqueName: \"kubernetes.io/projected/d0c235a1-d344-43b0-8a69-ce4e34f23de3-kube-api-access-xjzjr\") pod \"network-check-source-7cbc45fddc-kz5np\" (UID: \"d0c235a1-d344-43b0-8a69-ce4e34f23de3\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.705997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b17ce39-8f2d-4496-94d3-b62b3c984857-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f85d3569-b8be-47de-894a-7f2141464023-serving-cert\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706144    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.706587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdh42\" (UniqueName: \"kubernetes.io/projected/c19150e6-187a-4e99-907e-b584d8b3aa55-kube-api-access-rdh42\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38dfae07-7770-480a-941b-473232de5af7-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-certificates\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b17ce39-8f2d-4496-94d3-b62b3c984857-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-trusted-ca\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b17ce39-8f2d-4496-94d3-b62b3c984857-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706913    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4b17ce39-8f2d-4496-94d3-b62b3c984857-snapshots\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.706983    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b17ce39-8f2d-4496-94d3-b62b3c984857-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.707766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.707338    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-certificates\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.710199 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.710154    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38dfae07-7770-480a-941b-473232de5af7-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.710347 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.710324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b17ce39-8f2d-4496-94d3-b62b3c984857-serving-cert\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.710428 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.710378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-image-registry-private-configuration\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.710600 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.710579    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtq6m\" (UniqueName: \"kubernetes.io/projected/439cdf3d-8f3f-43a1-bfe7-8609d7456576-kube-api-access-qtq6m\") pod \"network-check-target-xp6p5\" (UID: \"439cdf3d-8f3f-43a1-bfe7-8609d7456576\") " pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:52.710962 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.710940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-installation-pull-secrets\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.714015 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.713996    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nt78\" (UniqueName: \"kubernetes.io/projected/4b17ce39-8f2d-4496-94d3-b62b3c984857-kube-api-access-5nt78\") pod \"insights-operator-6f9c644dd9-tm2tl\" (UID: \"4b17ce39-8f2d-4496-94d3-b62b3c984857\") " pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.715151 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.715127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmxm\" (UniqueName: \"kubernetes.io/projected/38dfae07-7770-480a-941b-473232de5af7-kube-api-access-cbmxm\") pod \"kube-storage-version-migrator-operator-55889f7b47-nht6h\" (UID: \"38dfae07-7770-480a-941b-473232de5af7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.715240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.715223    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-bound-sa-token\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.715290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.715238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb5vn\" (UniqueName: \"kubernetes.io/projected/2c502493-7814-4874-bac9-96021e28557d-kube-api-access-pb5vn\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:52.716189 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.716164    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj4p9\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-kube-api-access-jj4p9\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:52.789275 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.789242    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"
Sep 18 18:00:52.807294 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c19150e6-187a-4e99-907e-b584d8b3aa55-serving-cert\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.807417 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f85d3569-b8be-47de-894a-7f2141464023-config\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.807477 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82d574cd-7c7e-48d3-9c56-af1bb63402b4-cert\") pod \"ingress-canary-bhppq\" (UID: \"82d574cd-7c7e-48d3-9c56-af1bb63402b4\") " pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.807528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.807657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807532    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjzjr\" (UniqueName: \"kubernetes.io/projected/d0c235a1-d344-43b0-8a69-ce4e34f23de3-kube-api-access-xjzjr\") pod \"network-check-source-7cbc45fddc-kz5np\" (UID: \"d0c235a1-d344-43b0-8a69-ce4e34f23de3\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"
Sep 18 18:00:52.807657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.807657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f85d3569-b8be-47de-894a-7f2141464023-serving-cert\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.807657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdh42\" (UniqueName: \"kubernetes.io/projected/c19150e6-187a-4e99-907e-b584d8b3aa55-kube-api-access-rdh42\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a3558a4-49fc-4e96-986f-c1a7821da11a-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807731    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c19150e6-187a-4e99-907e-b584d8b3aa55-config\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4qqn\" (UniqueName: \"kubernetes.io/projected/f85d3569-b8be-47de-894a-7f2141464023-kube-api-access-t4qqn\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/6a3558a4-49fc-4e96-986f-c1a7821da11a-kube-api-access-2jl5w\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.807806    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c19150e6-187a-4e99-907e-b584d8b3aa55-trusted-ca\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.807886 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.807879    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls podName:6a3558a4-49fc-4e96-986f-c1a7821da11a nodeName:}" failed. No retries permitted until 2026-09-18 18:00:53.307860724 +0000 UTC m=+36.735073476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-hxhc4" (UID: "6a3558a4-49fc-4e96-986f-c1a7821da11a") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:52.808296 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807892    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f85d3569-b8be-47de-894a-7f2141464023-config\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.808296 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807898    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/995ff60d-7dd5-491a-aca8-34ce1216a8bc-tmp-dir\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.808296 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.807950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2q68\" (UniqueName: \"kubernetes.io/projected/995ff60d-7dd5-491a-aca8-34ce1216a8bc-kube-api-access-j2q68\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.808296 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.807971    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:00:52.808296 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.808014    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:53.308000121 +0000 UTC m=+36.735212865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : secret "router-metrics-certs-default" not found
Sep 18 18:00:52.808296 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.808228    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:53.308211141 +0000 UTC m=+36.735423901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : configmap references non-existent config key: service-ca.crt
Sep 18 18:00:52.808840 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpwg\" (UniqueName: \"kubernetes.io/projected/65231b2f-8527-44cc-9bc1-6629067f9311-kube-api-access-6mpwg\") pod \"volume-data-source-validator-5c4db8bdf9-ddlj2\" (UID: \"65231b2f-8527-44cc-9bc1-6629067f9311\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"
Sep 18 18:00:52.808945 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.808945 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-default-certificate\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.808945 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-stats-auth\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.809133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a3558a4-49fc-4e96-986f-c1a7821da11a-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.809133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvjc4\" (UniqueName: \"kubernetes.io/projected/82d574cd-7c7e-48d3-9c56-af1bb63402b4-kube-api-access-fvjc4\") pod \"ingress-canary-bhppq\" (UID: \"82d574cd-7c7e-48d3-9c56-af1bb63402b4\") " pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.809133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.808984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c19150e6-187a-4e99-907e-b584d8b3aa55-config\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.809133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.809002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c19150e6-187a-4e99-907e-b584d8b3aa55-trusted-ca\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.809355 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.809147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/995ff60d-7dd5-491a-aca8-34ce1216a8bc-config-volume\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.809355 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.809222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkg8x\" (UniqueName: \"kubernetes.io/projected/ca87bb38-f503-4312-8ea0-ea8941f1ea67-kube-api-access-xkg8x\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.811021 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.810999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c19150e6-187a-4e99-907e-b584d8b3aa55-serving-cert\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.811495 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.811478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-stats-auth\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.811576 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.811511    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-default-certificate\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.816262 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.816240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl"
Sep 18 18:00:52.817011 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.816958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjzjr\" (UniqueName: \"kubernetes.io/projected/d0c235a1-d344-43b0-8a69-ce4e34f23de3-kube-api-access-xjzjr\") pod \"network-check-source-7cbc45fddc-kz5np\" (UID: \"d0c235a1-d344-43b0-8a69-ce4e34f23de3\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"
Sep 18 18:00:52.817491 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.817447    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4qqn\" (UniqueName: \"kubernetes.io/projected/f85d3569-b8be-47de-894a-7f2141464023-kube-api-access-t4qqn\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.817491 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.817471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpwg\" (UniqueName: \"kubernetes.io/projected/65231b2f-8527-44cc-9bc1-6629067f9311-kube-api-access-6mpwg\") pod \"volume-data-source-validator-5c4db8bdf9-ddlj2\" (UID: \"65231b2f-8527-44cc-9bc1-6629067f9311\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"
Sep 18 18:00:52.817947 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.817929    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jl5w\" (UniqueName: \"kubernetes.io/projected/6a3558a4-49fc-4e96-986f-c1a7821da11a-kube-api-access-2jl5w\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:52.818113 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.818071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdh42\" (UniqueName: \"kubernetes.io/projected/c19150e6-187a-4e99-907e-b584d8b3aa55-kube-api-access-rdh42\") pod \"console-operator-fd4bb46c-w8lzm\" (UID: \"c19150e6-187a-4e99-907e-b584d8b3aa55\") " pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.819396 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.819378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkg8x\" (UniqueName: \"kubernetes.io/projected/ca87bb38-f503-4312-8ea0-ea8941f1ea67-kube-api-access-xkg8x\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:52.843695 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.843470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f85d3569-b8be-47de-894a-7f2141464023-serving-cert\") pod \"service-ca-operator-57cdc4b768-x2ff4\" (UID: \"f85d3569-b8be-47de-894a-7f2141464023\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.858141 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.858111    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:00:52.880201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.880129    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"
Sep 18 18:00:52.903217 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.903010    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"
Sep 18 18:00:52.910178 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910155    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82d574cd-7c7e-48d3-9c56-af1bb63402b4-cert\") pod \"ingress-canary-bhppq\" (UID: \"82d574cd-7c7e-48d3-9c56-af1bb63402b4\") " pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.910317 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/995ff60d-7dd5-491a-aca8-34ce1216a8bc-tmp-dir\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.910385 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2q68\" (UniqueName: \"kubernetes.io/projected/995ff60d-7dd5-491a-aca8-34ce1216a8bc-kube-api-access-j2q68\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.910385 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.910490 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvjc4\" (UniqueName: \"kubernetes.io/projected/82d574cd-7c7e-48d3-9c56-af1bb63402b4-kube-api-access-fvjc4\") pod \"ingress-canary-bhppq\" (UID: \"82d574cd-7c7e-48d3-9c56-af1bb63402b4\") " pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.910490 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/995ff60d-7dd5-491a-aca8-34ce1216a8bc-config-volume\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.911223 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.910981    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/995ff60d-7dd5-491a-aca8-34ce1216a8bc-config-volume\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.911223 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.910986    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:00:52.911223 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:52.911097    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls podName:995ff60d-7dd5-491a-aca8-34ce1216a8bc nodeName:}" failed. No retries permitted until 2026-09-18 18:00:53.411076974 +0000 UTC m=+36.838289720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls") pod "dns-default-vwp4c" (UID: "995ff60d-7dd5-491a-aca8-34ce1216a8bc") : secret "dns-default-metrics-tls" not found
Sep 18 18:00:52.911223 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.911169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/995ff60d-7dd5-491a-aca8-34ce1216a8bc-tmp-dir\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.913994 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.913956    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82d574cd-7c7e-48d3-9c56-af1bb63402b4-cert\") pod \"ingress-canary-bhppq\" (UID: \"82d574cd-7c7e-48d3-9c56-af1bb63402b4\") " pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.922507 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.922461    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvjc4\" (UniqueName: \"kubernetes.io/projected/82d574cd-7c7e-48d3-9c56-af1bb63402b4-kube-api-access-fvjc4\") pod \"ingress-canary-bhppq\" (UID: \"82d574cd-7c7e-48d3-9c56-af1bb63402b4\") " pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:52.924714 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.924675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2q68\" (UniqueName: \"kubernetes.io/projected/995ff60d-7dd5-491a-aca8-34ce1216a8bc-kube-api-access-j2q68\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:52.950993 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.950957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"
Sep 18 18:00:52.970916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:52.970867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pdzvz_b0445beb-a1ca-4b60-be4a-8f040edeab39/konnectivity-agent/0.log" limit=1024
Sep 18 18:00:53.003787 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.003601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bhppq"
Sep 18 18:00:53.098617 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.098535    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h"]
Sep 18 18:00:53.101580 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.101557    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dfae07_7770_480a_941b_473232de5af7.slice/crio-3c61a55b262194b58ebf728c2e59332eac63798005d29cba487b67d8fa1049ca WatchSource:0}: Error finding container 3c61a55b262194b58ebf728c2e59332eac63798005d29cba487b67d8fa1049ca: Status 404 returned error can't find the container with id 3c61a55b262194b58ebf728c2e59332eac63798005d29cba487b67d8fa1049ca
Sep 18 18:00:53.115155 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.115134    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-tm2tl"]
Sep 18 18:00:53.117707 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.117685    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b17ce39_8f2d_4496_94d3_b62b3c984857.slice/crio-2e0d2b904cc6e4363780455725da4731a7cfe1237102c0e373bf56fae5592757 WatchSource:0}: Error finding container 2e0d2b904cc6e4363780455725da4731a7cfe1237102c0e373bf56fae5592757: Status 404 returned error can't find the container with id 2e0d2b904cc6e4363780455725da4731a7cfe1237102c0e373bf56fae5592757
Sep 18 18:00:53.182007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.181972    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-w8lzm"]
Sep 18 18:00:53.185811 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.185768    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19150e6_187a_4e99_907e_b584d8b3aa55.slice/crio-e4e2dd5df888c4f49eaecdba0e77fb74b1398cc39be3bb430c612fc8924f85d7 WatchSource:0}: Error finding container e4e2dd5df888c4f49eaecdba0e77fb74b1398cc39be3bb430c612fc8924f85d7: Status 404 returned error can't find the container with id e4e2dd5df888c4f49eaecdba0e77fb74b1398cc39be3bb430c612fc8924f85d7
Sep 18 18:00:53.186641 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.186611    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4"]
Sep 18 18:00:53.191543 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.191508    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85d3569_b8be_47de_894a_7f2141464023.slice/crio-bc02724e49dfd95c13b18b944572c79a6751ddad43f3edc701790be2ccd9633c WatchSource:0}: Error finding container bc02724e49dfd95c13b18b944572c79a6751ddad43f3edc701790be2ccd9633c: Status 404 returned error can't find the container with id bc02724e49dfd95c13b18b944572c79a6751ddad43f3edc701790be2ccd9633c
Sep 18 18:00:53.203859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.203837    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2"]
Sep 18 18:00:53.206520 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.206497    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65231b2f_8527_44cc_9bc1_6629067f9311.slice/crio-ba326324d8ef4fad15b7515ea4bf6cb8d8a376ad0c1c88bcadd18e9d0f0084da WatchSource:0}: Error finding container ba326324d8ef4fad15b7515ea4bf6cb8d8a376ad0c1c88bcadd18e9d0f0084da: Status 404 returned error can't find the container with id ba326324d8ef4fad15b7515ea4bf6cb8d8a376ad0c1c88bcadd18e9d0f0084da
Sep 18 18:00:53.213321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.213293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:53.213394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.213370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:53.213457 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.213437    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:00:53.213506 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.213493    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:00:53.213543 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.213508    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76756c9684-jxrnf: secret "image-registry-tls" not found
Sep 18 18:00:53.213582 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.213497    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls podName:2c502493-7814-4874-bac9-96021e28557d nodeName:}" failed. No retries permitted until 2026-09-18 18:00:54.21347854 +0000 UTC m=+37.640691272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gp99n" (UID: "2c502493-7814-4874-bac9-96021e28557d") : secret "samples-operator-tls" not found
Sep 18 18:00:53.213582 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.213567    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls podName:c3a3834a-cc44-470d-8af5-665d4a6e5463 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:54.213552017 +0000 UTC m=+37.640764753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls") pod "image-registry-76756c9684-jxrnf" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463") : secret "image-registry-tls" not found
Sep 18 18:00:53.251473 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.251448    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np"]
Sep 18 18:00:53.254610 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.254574    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c235a1_d344_43b0_8a69_ce4e34f23de3.slice/crio-b0bdc48f2aefd9b415df145c7f98677d9fdee2bd6cc65c61e44ab9030726d3e0 WatchSource:0}: Error finding container b0bdc48f2aefd9b415df145c7f98677d9fdee2bd6cc65c61e44ab9030726d3e0: Status 404 returned error can't find the container with id b0bdc48f2aefd9b415df145c7f98677d9fdee2bd6cc65c61e44ab9030726d3e0
Sep 18 18:00:53.304184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.304150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bhppq"]
Sep 18 18:00:53.306735 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:53.306708    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d574cd_7c7e_48d3_9c56_af1bb63402b4.slice/crio-197e96101b2de5122bfac8af7d64ce08e7966c8d5c39f47e7a17804e61bc9434 WatchSource:0}: Error finding container 197e96101b2de5122bfac8af7d64ce08e7966c8d5c39f47e7a17804e61bc9434: Status 404 returned error can't find the container with id 197e96101b2de5122bfac8af7d64ce08e7966c8d5c39f47e7a17804e61bc9434
Sep 18 18:00:53.314485 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.314464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:53.314567 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.314494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:53.314632 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.314581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:53.314685 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.314626    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:00:53.314685 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.314674    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:53.314773 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.314696    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:54.314675674 +0000 UTC m=+37.741888408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : secret "router-metrics-certs-default" not found
Sep 18 18:00:53.314773 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.314716    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls podName:6a3558a4-49fc-4e96-986f-c1a7821da11a nodeName:}" failed. No retries permitted until 2026-09-18 18:00:54.314703838 +0000 UTC m=+37.741916570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-hxhc4" (UID: "6a3558a4-49fc-4e96-986f-c1a7821da11a") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:53.314773 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.314729    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:54.314722392 +0000 UTC m=+37.741935124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : configmap references non-existent config key: service-ca.crt
Sep 18 18:00:53.415576 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.415503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:53.415715 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.415668    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:00:53.415772 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:53.415744    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls podName:995ff60d-7dd5-491a-aca8-34ce1216a8bc nodeName:}" failed. No retries permitted until 2026-09-18 18:00:54.415723662 +0000 UTC m=+37.842936409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls") pod "dns-default-vwp4c" (UID: "995ff60d-7dd5-491a-aca8-34ce1216a8bc") : secret "dns-default-metrics-tls" not found
Sep 18 18:00:53.913927 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.913892    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:00:53.914474 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.914458    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:00:53.915223 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.915201    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:53.918281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.918256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fqhj6\""
Sep 18 18:00:53.918527 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.918509    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:00:53.919214 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.918698    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:00:53.919214 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.918871    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kxcq\""
Sep 18 18:00:53.952040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:53.952019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:00:54.082336 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.082201    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np" event={"ID":"d0c235a1-d344-43b0-8a69-ce4e34f23de3","Type":"ContainerStarted","Data":"b0bdc48f2aefd9b415df145c7f98677d9fdee2bd6cc65c61e44ab9030726d3e0"}
Sep 18 18:00:54.084515 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.084435    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2" event={"ID":"65231b2f-8527-44cc-9bc1-6629067f9311","Type":"ContainerStarted","Data":"ba326324d8ef4fad15b7515ea4bf6cb8d8a376ad0c1c88bcadd18e9d0f0084da"}
Sep 18 18:00:54.087187 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.087161    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4" event={"ID":"f85d3569-b8be-47de-894a-7f2141464023","Type":"ContainerStarted","Data":"bc02724e49dfd95c13b18b944572c79a6751ddad43f3edc701790be2ccd9633c"}
Sep 18 18:00:54.089612 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.089540    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" event={"ID":"c19150e6-187a-4e99-907e-b584d8b3aa55","Type":"ContainerStarted","Data":"e4e2dd5df888c4f49eaecdba0e77fb74b1398cc39be3bb430c612fc8924f85d7"}
Sep 18 18:00:54.095094 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.095018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl" event={"ID":"4b17ce39-8f2d-4496-94d3-b62b3c984857","Type":"ContainerStarted","Data":"2e0d2b904cc6e4363780455725da4731a7cfe1237102c0e373bf56fae5592757"}
Sep 18 18:00:54.097843 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.097777    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bhppq" event={"ID":"82d574cd-7c7e-48d3-9c56-af1bb63402b4","Type":"ContainerStarted","Data":"197e96101b2de5122bfac8af7d64ce08e7966c8d5c39f47e7a17804e61bc9434"}
Sep 18 18:00:54.099326 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.099266    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h" event={"ID":"38dfae07-7770-480a-941b-473232de5af7","Type":"ContainerStarted","Data":"3c61a55b262194b58ebf728c2e59332eac63798005d29cba487b67d8fa1049ca"}
Sep 18 18:00:54.224666 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.224571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:54.224666 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.224658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:54.224923 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.224823    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:00:54.224923 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.224839    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76756c9684-jxrnf: secret "image-registry-tls" not found
Sep 18 18:00:54.224923 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.224907    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:00:54.224923 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.224919    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls podName:c3a3834a-cc44-470d-8af5-665d4a6e5463 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.224898072 +0000 UTC m=+39.652110818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls") pod "image-registry-76756c9684-jxrnf" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463") : secret "image-registry-tls" not found
Sep 18 18:00:54.225205 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.224955    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls podName:2c502493-7814-4874-bac9-96021e28557d nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.224945287 +0000 UTC m=+39.652158019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gp99n" (UID: "2c502493-7814-4874-bac9-96021e28557d") : secret "samples-operator-tls" not found
Sep 18 18:00:54.256862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.256792    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xp6p5"]
Sep 18 18:00:54.298719 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.297531    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"]
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.325867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.325915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.325946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.326206    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.326271    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls podName:6a3558a4-49fc-4e96-986f-c1a7821da11a nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.326253779 +0000 UTC m=+39.753466516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-hxhc4" (UID: "6a3558a4-49fc-4e96-986f-c1a7821da11a") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.326690    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.326672158 +0000 UTC m=+39.753884894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : configmap references non-existent config key: service-ca.crt
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.326761    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:00:54.326831 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.326797    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.326785604 +0000 UTC m=+39.753998344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : secret "router-metrics-certs-default" not found
Sep 18 18:00:54.337122 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.336624    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"]
Sep 18 18:00:54.337122 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.336732    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:54.340632 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.339292    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 18 18:00:54.340632 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.339611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7gpcf\""
Sep 18 18:00:54.340632 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.339805    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 18 18:00:54.428010 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.427293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:54.428010 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.427347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:54.428010 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.427419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:54.428010 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.427613    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:00:54.428010 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.427674    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls podName:995ff60d-7dd5-491a-aca8-34ce1216a8bc nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.427655914 +0000 UTC m=+39.854868650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls") pod "dns-default-vwp4c" (UID: "995ff60d-7dd5-491a-aca8-34ce1216a8bc") : secret "dns-default-metrics-tls" not found
Sep 18 18:00:54.528808 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.528690    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:54.528808 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.528798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:54.529491 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.529175    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:00:54.529491 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:54.529252    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert podName:b2b950ab-bdf2-4d50-8467-d4d2e762af1f nodeName:}" failed. No retries permitted until 2026-09-18 18:00:55.029230839 +0000 UTC m=+38.456443577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-g47n6" (UID: "b2b950ab-bdf2-4d50-8467-d4d2e762af1f") : secret "networking-console-plugin-cert" not found
Sep 18 18:00:54.529726 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:54.529677    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:54.927018 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:00:54.926982    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439cdf3d_8f3f_43a1_bfe7_8609d7456576.slice/crio-26ba205d5ffccda03c65b09bdd21551d0a6a996e36cea1557ede256f63295755 WatchSource:0}: Error finding container 26ba205d5ffccda03c65b09bdd21551d0a6a996e36cea1557ede256f63295755: Status 404 returned error can't find the container with id 26ba205d5ffccda03c65b09bdd21551d0a6a996e36cea1557ede256f63295755
Sep 18 18:00:55.033995 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:55.033180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:55.033995 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:55.033566    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:00:55.033995 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:55.033632    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert podName:b2b950ab-bdf2-4d50-8467-d4d2e762af1f nodeName:}" failed. No retries permitted until 2026-09-18 18:00:56.03361344 +0000 UTC m=+39.460826196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-g47n6" (UID: "b2b950ab-bdf2-4d50-8467-d4d2e762af1f") : secret "networking-console-plugin-cert" not found
Sep 18 18:00:55.106522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:55.106453    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xp6p5" event={"ID":"439cdf3d-8f3f-43a1-bfe7-8609d7456576","Type":"ContainerStarted","Data":"26ba205d5ffccda03c65b09bdd21551d0a6a996e36cea1557ede256f63295755"}
Sep 18 18:00:56.047257 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.047163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:56.047797 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.047325    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:00:56.047797 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.047404    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert podName:b2b950ab-bdf2-4d50-8467-d4d2e762af1f nodeName:}" failed. No retries permitted until 2026-09-18 18:00:58.047382072 +0000 UTC m=+41.474594825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-g47n6" (UID: "b2b950ab-bdf2-4d50-8467-d4d2e762af1f") : secret "networking-console-plugin-cert" not found
Sep 18 18:00:56.249611 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.249566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:00:56.249778 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.249696    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:00:56.249885 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.249856    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:00:56.249885 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.249880    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76756c9684-jxrnf: secret "image-registry-tls" not found
Sep 18 18:00:56.250074 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.249940    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls podName:c3a3834a-cc44-470d-8af5-665d4a6e5463 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:00.249922481 +0000 UTC m=+43.677135232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls") pod "image-registry-76756c9684-jxrnf" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463") : secret "image-registry-tls" not found
Sep 18 18:00:56.250415 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.250390    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:00:56.250510 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.250455    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls podName:2c502493-7814-4874-bac9-96021e28557d nodeName:}" failed. No retries permitted until 2026-09-18 18:01:00.250438793 +0000 UTC m=+43.677651539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gp99n" (UID: "2c502493-7814-4874-bac9-96021e28557d") : secret "samples-operator-tls" not found
Sep 18 18:00:56.350222 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.350132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:56.350222 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.350192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:00:56.350222 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.350211    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:00:56.350507 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.350488    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:56.350568 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.350562    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls podName:6a3558a4-49fc-4e96-986f-c1a7821da11a nodeName:}" failed. No retries permitted until 2026-09-18 18:01:00.350547723 +0000 UTC m=+43.777760455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-hxhc4" (UID: "6a3558a4-49fc-4e96-986f-c1a7821da11a") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:00:56.350968 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.350945    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:00.350926522 +0000 UTC m=+43.778139253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : configmap references non-existent config key: service-ca.crt
Sep 18 18:00:56.351084 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.351030    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:00:56.351084 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.351072    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:00.351063296 +0000 UTC m=+43.778276030 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : secret "router-metrics-certs-default" not found
Sep 18 18:00:56.450986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:56.450952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:00:56.451193 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.451094    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:00:56.451193 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:56.451148    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls podName:995ff60d-7dd5-491a-aca8-34ce1216a8bc nodeName:}" failed. No retries permitted until 2026-09-18 18:01:00.451134192 +0000 UTC m=+43.878346924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls") pod "dns-default-vwp4c" (UID: "995ff60d-7dd5-491a-aca8-34ce1216a8bc") : secret "dns-default-metrics-tls" not found
Sep 18 18:00:58.066848 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:00:58.066722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:00:58.067317 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:58.066899    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:00:58.067317 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:00:58.066975    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert podName:b2b950ab-bdf2-4d50-8467-d4d2e762af1f nodeName:}" failed. No retries permitted until 2026-09-18 18:01:02.066955734 +0000 UTC m=+45.494168489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-g47n6" (UID: "b2b950ab-bdf2-4d50-8467-d4d2e762af1f") : secret "networking-console-plugin-cert" not found
Sep 18 18:01:00.285496 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:00.285309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:01:00.285949 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.285471    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:01:00.285949 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.285610    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls podName:2c502493-7814-4874-bac9-96021e28557d nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.28559457 +0000 UTC m=+51.712807318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gp99n" (UID: "2c502493-7814-4874-bac9-96021e28557d") : secret "samples-operator-tls" not found
Sep 18 18:01:00.285949 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:00.285545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:00.285949 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.285636    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:00.285949 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.285652    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76756c9684-jxrnf: secret "image-registry-tls" not found
Sep 18 18:01:00.285949 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.285691    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls podName:c3a3834a-cc44-470d-8af5-665d4a6e5463 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.285684004 +0000 UTC m=+51.712896736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls") pod "image-registry-76756c9684-jxrnf" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463") : secret "image-registry-tls" not found
Sep 18 18:01:00.386767 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:00.386725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:00.386942 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:00.386772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:00.386942 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:00.386845    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:01:00.386942 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.386913    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.386894815 +0000 UTC m=+51.814107551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : configmap references non-existent config key: service-ca.crt
Sep 18 18:01:00.387136 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.386960    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:01:00.387136 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.386998    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:01:00.387136 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.387025    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls podName:6a3558a4-49fc-4e96-986f-c1a7821da11a nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.387009749 +0000 UTC m=+51.814222485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-hxhc4" (UID: "6a3558a4-49fc-4e96-986f-c1a7821da11a") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:01:00.387136 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.387063    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.387035479 +0000 UTC m=+51.814248216 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : secret "router-metrics-certs-default" not found
Sep 18 18:01:00.487972 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:00.487935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:00.488162 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.488116    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:00.488244 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:00.488195    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls podName:995ff60d-7dd5-491a-aca8-34ce1216a8bc nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.488168275 +0000 UTC m=+51.915381031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls") pod "dns-default-vwp4c" (UID: "995ff60d-7dd5-491a-aca8-34ce1216a8bc") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:02.102383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:02.102340    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:01:02.102773 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:02.102461    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:01:02.102773 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:02.102549    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert podName:b2b950ab-bdf2-4d50-8467-d4d2e762af1f nodeName:}" failed. No retries permitted until 2026-09-18 18:01:10.102526685 +0000 UTC m=+53.529739422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-g47n6" (UID: "b2b950ab-bdf2-4d50-8467-d4d2e762af1f") : secret "networking-console-plugin-cert" not found
Sep 18 18:01:04.135109 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.135072    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np" event={"ID":"d0c235a1-d344-43b0-8a69-ce4e34f23de3","Type":"ContainerStarted","Data":"84c68c6b3e3f58129ca7254431f9c5b86633754a51521228dcfbcea7f090f0cf"}
Sep 18 18:01:04.136858 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.136833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2" event={"ID":"65231b2f-8527-44cc-9bc1-6629067f9311","Type":"ContainerStarted","Data":"7e018d3a829c7f5af32a08872d0ed9b46ad40034ed876e57bde0063076f57dfd"}
Sep 18 18:01:04.140326 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.140301    2582 generic.go:356] "Generic (PLEG): container finished" podID="0497d646-848f-4d71-b54c-db65e05579ee" containerID="19f88143b450251b253b2c62cdd5a00391a48c35b302254601fcfd9044c1e5f4" exitCode=0
Sep 18 18:01:04.140423 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.140379    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerDied","Data":"19f88143b450251b253b2c62cdd5a00391a48c35b302254601fcfd9044c1e5f4"}
Sep 18 18:01:04.142741 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.142718    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4" event={"ID":"f85d3569-b8be-47de-894a-7f2141464023","Type":"ContainerStarted","Data":"f00f9365c68782bfdb130daf29c04ca837fee830b6d3aa66061cba263d421d87"}
Sep 18 18:01:04.145968 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.145949    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/0.log"
Sep 18 18:01:04.146063 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.145991    2582 generic.go:356] "Generic (PLEG): container finished" podID="c19150e6-187a-4e99-907e-b584d8b3aa55" containerID="16eac95ab1a599de87ba45a7591280f729eb0eac3d003acc9c3780f68381b1b7" exitCode=255
Sep 18 18:01:04.146112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.146083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" event={"ID":"c19150e6-187a-4e99-907e-b584d8b3aa55","Type":"ContainerDied","Data":"16eac95ab1a599de87ba45a7591280f729eb0eac3d003acc9c3780f68381b1b7"}
Sep 18 18:01:04.146290 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.146257    2582 scope.go:117] "RemoveContainer" containerID="16eac95ab1a599de87ba45a7591280f729eb0eac3d003acc9c3780f68381b1b7"
Sep 18 18:01:04.149919 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.149888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl" event={"ID":"4b17ce39-8f2d-4496-94d3-b62b3c984857","Type":"ContainerStarted","Data":"4d11d91a77b6d500b35381281df13c63a0c0d5b6ced3159ca0bdb20eb9c4cf75"}
Sep 18 18:01:04.151976 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.151930    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-kz5np" podStartSLOduration=28.32219205 podStartE2EDuration="38.151914061s" totalImagesPullingTime="9.829722011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.150519722 +0000 UTC m=+47.577732477" watchObservedRunningTime="2026-09-18 18:01:04.151914061 +0000 UTC m=+47.579126813"
Sep 18 18:01:04.154408 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.154385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bhppq" event={"ID":"82d574cd-7c7e-48d3-9c56-af1bb63402b4","Type":"ContainerStarted","Data":"df6d235a37193df9a5c23cdcdf2cd4449de63632bd5726e3b68f7b75376c7654"}
Sep 18 18:01:04.158176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.157095    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h" event={"ID":"38dfae07-7770-480a-941b-473232de5af7","Type":"ContainerStarted","Data":"9e9d57bf42f811ef902b22ccfbeb090ad6161940104c886e41dfa5c766f0be7a"}
Sep 18 18:01:04.162407 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.162383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xp6p5" event={"ID":"439cdf3d-8f3f-43a1-bfe7-8609d7456576","Type":"ContainerStarted","Data":"730fb5d6ff77759ddd49b1b114736eb1a399c8e3694e3eccf2f7673941409663"}
Sep 18 18:01:04.162652 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.162639    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:01:04.211377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.211335    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl" podStartSLOduration=28.270891061 podStartE2EDuration="38.211317225s" totalImagesPullingTime="9.940426164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.210940326 +0000 UTC m=+47.638153082" watchObservedRunningTime="2026-09-18 18:01:04.211317225 +0000 UTC m=+47.638529977"
Sep 18 18:01:04.232639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.231953    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4" podStartSLOduration=28.337043952 podStartE2EDuration="38.231934179s" totalImagesPullingTime="9.894890227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.231362121 +0000 UTC m=+47.658574876" watchObservedRunningTime="2026-09-18 18:01:04.231934179 +0000 UTC m=+47.659146935"
Sep 18 18:01:04.273553 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.270139    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-ddlj2" podStartSLOduration=28.597792037 podStartE2EDuration="38.270118676s" totalImagesPullingTime="9.672326639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.251115839 +0000 UTC m=+47.678328597" watchObservedRunningTime="2026-09-18 18:01:04.270118676 +0000 UTC m=+47.697331432"
Sep 18 18:01:04.297136 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.296687    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xp6p5" podStartSLOduration=37.131856437 podStartE2EDuration="45.296667539s" totalImagesPullingTime="8.164811102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.296397618 +0000 UTC m=+47.723610373" watchObservedRunningTime="2026-09-18 18:01:04.296667539 +0000 UTC m=+47.723880292"
Sep 18 18:01:04.298810 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.297349    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h" podStartSLOduration=28.314710875 podStartE2EDuration="38.297329374s" totalImagesPullingTime="9.982618499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.271612563 +0000 UTC m=+47.698825318" watchObservedRunningTime="2026-09-18 18:01:04.297329374 +0000 UTC m=+47.724542129"
Sep 18 18:01:04.322635 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.322583    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bhppq" podStartSLOduration=2.541259361 podStartE2EDuration="12.32256125s" totalImagesPullingTime="9.781301889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:04.321258035 +0000 UTC m=+47.748470791" watchObservedRunningTime="2026-09-18 18:01:04.32256125 +0000 UTC m=+47.749774005"
Sep 18 18:01:04.344336 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.343003    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s2k9s"]
Sep 18 18:01:04.345622 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.345599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.349472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.349084    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dd246\""
Sep 18 18:01:04.349472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.349310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 18:01:04.349472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.349385    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 18:01:04.356036 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.355991    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2k9s"]
Sep 18 18:01:04.529310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.529268    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.529434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.529417    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4965f0d2-8952-4a08-b640-541fa3e20363-data-volume\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.529504 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.529453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.529504 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.529482    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4965f0d2-8952-4a08-b640-541fa3e20363-crio-socket\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.529643 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.529619    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hd54\" (UniqueName: \"kubernetes.io/projected/4965f0d2-8952-4a08-b640-541fa3e20363-kube-api-access-9hd54\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631090 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631241 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4965f0d2-8952-4a08-b640-541fa3e20363-data-volume\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631241 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631241 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:04.631211    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:04.631241 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4965f0d2-8952-4a08-b640-541fa3e20363-crio-socket\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631382 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:04.631281    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:05.131264055 +0000 UTC m=+48.558476820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:04.631382 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631311    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hd54\" (UniqueName: \"kubernetes.io/projected/4965f0d2-8952-4a08-b640-541fa3e20363-kube-api-access-9hd54\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631382 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:04.631326    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:04.631490 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:04.631409    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:05.131390206 +0000 UTC m=+48.558602950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:04.631490 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631421    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4965f0d2-8952-4a08-b640-541fa3e20363-crio-socket\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.631574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.631555    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4965f0d2-8952-4a08-b640-541fa3e20363-data-volume\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:04.641503 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:04.641478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hd54\" (UniqueName: \"kubernetes.io/projected/4965f0d2-8952-4a08-b640-541fa3e20363-kube-api-access-9hd54\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:05.135160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.135122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:05.135515 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.135230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:05.135515 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:05.135269    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:05.135515 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:05.135323    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:05.135515 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:05.135343    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:06.135327201 +0000 UTC m=+49.562539950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:05.135515 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:05.135378    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:06.135366285 +0000 UTC m=+49.562579030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:05.167995 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.167974    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:01:05.168404 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.168390    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/0.log"
Sep 18 18:01:05.168452 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.168427    2582 generic.go:356] "Generic (PLEG): container finished" podID="c19150e6-187a-4e99-907e-b584d8b3aa55" containerID="a96bbe4cdf5e19d51ec92ef391902c687fb112306667e1e4dd5488ad9931877f" exitCode=255
Sep 18 18:01:05.168526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.168506    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" event={"ID":"c19150e6-187a-4e99-907e-b584d8b3aa55","Type":"ContainerDied","Data":"a96bbe4cdf5e19d51ec92ef391902c687fb112306667e1e4dd5488ad9931877f"}
Sep 18 18:01:05.168578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.168559    2582 scope.go:117] "RemoveContainer" containerID="16eac95ab1a599de87ba45a7591280f729eb0eac3d003acc9c3780f68381b1b7"
Sep 18 18:01:05.168757 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.168743    2582 scope.go:117] "RemoveContainer" containerID="a96bbe4cdf5e19d51ec92ef391902c687fb112306667e1e4dd5488ad9931877f"
Sep 18 18:01:05.168947 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:05.168924    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-w8lzm_openshift-console-operator(c19150e6-187a-4e99-907e-b584d8b3aa55)\"" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" podUID="c19150e6-187a-4e99-907e-b584d8b3aa55"
Sep 18 18:01:05.171909 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.171887    2582 generic.go:356] "Generic (PLEG): container finished" podID="0497d646-848f-4d71-b54c-db65e05579ee" containerID="5b47239f7ebc2bcc0b01b9884a9e635fd83426f38e95f2e41d4ba76e17312f3e" exitCode=0
Sep 18 18:01:05.172028 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.172004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerDied","Data":"5b47239f7ebc2bcc0b01b9884a9e635fd83426f38e95f2e41d4ba76e17312f3e"}
Sep 18 18:01:05.724202 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:05.724171    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-trjfw_3dc8b275-9b48-462a-a66f-659147ccc56d/dns-node-resolver/0.log"
Sep 18 18:01:06.145577 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.145545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:06.145985 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.145624    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:06.145985 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:06.145665    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:06.145985 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:06.145733    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.145715738 +0000 UTC m=+51.572928492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:06.145985 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:06.145796    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:06.145985 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:06.145863    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.14584494 +0000 UTC m=+51.573057677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:06.180851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.180824    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" event={"ID":"0497d646-848f-4d71-b54c-db65e05579ee","Type":"ContainerStarted","Data":"e18bfb39cf1ab8a1284266bac9e927879c6cc099139b07fb16a1895f00c390a9"}
Sep 18 18:01:06.182438 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.182420    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:01:06.182687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.182675    2582 scope.go:117] "RemoveContainer" containerID="a96bbe4cdf5e19d51ec92ef391902c687fb112306667e1e4dd5488ad9931877f"
Sep 18 18:01:06.182816 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:06.182800    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-w8lzm_openshift-console-operator(c19150e6-187a-4e99-907e-b584d8b3aa55)\"" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" podUID="c19150e6-187a-4e99-907e-b584d8b3aa55"
Sep 18 18:01:06.202542 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.202505    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7sqh6" podStartSLOduration=8.436616734 podStartE2EDuration="47.202490829s" totalImagesPullingTime="38.765874095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:06.200437065 +0000 UTC m=+49.627649828" watchObservedRunningTime="2026-09-18 18:01:06.202490829 +0000 UTC m=+49.629703609"
Sep 18 18:01:06.324487 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:06.324462    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6fz2m_dba2c0a8-d33d-4dc7-9b6b-0ec277193214/node-ca/0.log"
Sep 18 18:01:07.123094 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:07.123039    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bhppq_82d574cd-7c7e-48d3-9c56-af1bb63402b4/serve-healthcheck-canary/0.log"
Sep 18 18:01:08.160552 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.160519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:08.160984 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.160633    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:08.160984 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.160642    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:08.160984 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.160713    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:12.160694167 +0000 UTC m=+55.587906918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:08.160984 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.160777    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:08.160984 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.160863    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:12.160848666 +0000 UTC m=+55.588061398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:08.362697 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.362663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:01:08.362854 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.362728    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:08.362854 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.362807    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 18:01:08.362854 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.362821    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:08.362854 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.362831    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76756c9684-jxrnf: secret "image-registry-tls" not found
Sep 18 18:01:08.362978 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.362879    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls podName:c3a3834a-cc44-470d-8af5-665d4a6e5463 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.362867042 +0000 UTC m=+67.790079774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls") pod "image-registry-76756c9684-jxrnf" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463") : secret "image-registry-tls" not found
Sep 18 18:01:08.362978 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.362891    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls podName:2c502493-7814-4874-bac9-96021e28557d nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.362885459 +0000 UTC m=+67.790098190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gp99n" (UID: "2c502493-7814-4874-bac9-96021e28557d") : secret "samples-operator-tls" not found
Sep 18 18:01:08.463359 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.463293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:08.463359 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.463338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:08.463501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.463365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:01:08.463501 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.463454    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 18:01:08.463501 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.463472    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 18:01:08.463501 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.463463    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.463446061 +0000 UTC m=+67.890658796 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : configmap references non-existent config key: service-ca.crt
Sep 18 18:01:08.463624 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.463510    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs podName:ca87bb38-f503-4312-8ea0-ea8941f1ea67 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.463498897 +0000 UTC m=+67.890711630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs") pod "router-default-598c64547d-8cwhs" (UID: "ca87bb38-f503-4312-8ea0-ea8941f1ea67") : secret "router-metrics-certs-default" not found
Sep 18 18:01:08.463624 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.463521    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls podName:6a3558a4-49fc-4e96-986f-c1a7821da11a nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.463514855 +0000 UTC m=+67.890727587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-hxhc4" (UID: "6a3558a4-49fc-4e96-986f-c1a7821da11a") : secret "cluster-monitoring-operator-tls" not found
Sep 18 18:01:08.563918 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:08.563891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:08.564072 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.563994    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:08.564072 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:08.564035    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls podName:995ff60d-7dd5-491a-aca8-34ce1216a8bc nodeName:}" failed. No retries permitted until 2026-09-18 18:01:24.564024531 +0000 UTC m=+67.991237263 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls") pod "dns-default-vwp4c" (UID: "995ff60d-7dd5-491a-aca8-34ce1216a8bc") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:10.176361 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:10.176326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:01:10.176725 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:10.176483    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 18:01:10.176725 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:10.176547    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert podName:b2b950ab-bdf2-4d50-8467-d4d2e762af1f nodeName:}" failed. No retries permitted until 2026-09-18 18:01:26.176531751 +0000 UTC m=+69.603744483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-g47n6" (UID: "b2b950ab-bdf2-4d50-8467-d4d2e762af1f") : secret "networking-console-plugin-cert" not found
Sep 18 18:01:12.192878 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:12.192840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:12.193278 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:12.192967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:12.193278 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:12.193004    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:12.193278 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:12.193108    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls podName:4965f0d2-8952-4a08-b640-541fa3e20363 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:20.193086544 +0000 UTC m=+63.620299277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2k9s" (UID: "4965f0d2-8952-4a08-b640-541fa3e20363") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:12.193593 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:12.193575    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4965f0d2-8952-4a08-b640-541fa3e20363-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:12.858913 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:12.858878    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:01:12.859095 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:12.858962    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:01:12.859294 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:12.859281    2582 scope.go:117] "RemoveContainer" containerID="a96bbe4cdf5e19d51ec92ef391902c687fb112306667e1e4dd5488ad9931877f"
Sep 18 18:01:12.859447 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:12.859428    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-w8lzm_openshift-console-operator(c19150e6-187a-4e99-907e-b584d8b3aa55)\"" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" podUID="c19150e6-187a-4e99-907e-b584d8b3aa55"
Sep 18 18:01:20.087299 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:20.087269    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hbxhg"
Sep 18 18:01:20.258979 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:20.258944    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:20.261409 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:20.261385    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4965f0d2-8952-4a08-b640-541fa3e20363-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2k9s\" (UID: \"4965f0d2-8952-4a08-b640-541fa3e20363\") " pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:20.342909 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:20.342841    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dd246\""
Sep 18 18:01:20.350745 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:20.350727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2k9s"
Sep 18 18:01:20.669624 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:20.669429    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2k9s"]
Sep 18 18:01:20.672147 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:20.672121    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4965f0d2_8952_4a08_b640_541fa3e20363.slice/crio-ce5ab7cece5a79f04677e5592cc02d2279dff8aadb0c9c04314519a36f7f01cd WatchSource:0}: Error finding container ce5ab7cece5a79f04677e5592cc02d2279dff8aadb0c9c04314519a36f7f01cd: Status 404 returned error can't find the container with id ce5ab7cece5a79f04677e5592cc02d2279dff8aadb0c9c04314519a36f7f01cd
Sep 18 18:01:21.236025 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:21.235985    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2k9s" event={"ID":"4965f0d2-8952-4a08-b640-541fa3e20363","Type":"ContainerStarted","Data":"3c22809acd466f01c93b317b649fc87cf7b0b9acc2777ce25c217bfb795ec152"}
Sep 18 18:01:21.236427 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:21.236070    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2k9s" event={"ID":"4965f0d2-8952-4a08-b640-541fa3e20363","Type":"ContainerStarted","Data":"ce5ab7cece5a79f04677e5592cc02d2279dff8aadb0c9c04314519a36f7f01cd"}
Sep 18 18:01:22.240787 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:22.240763    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2k9s" event={"ID":"4965f0d2-8952-4a08-b640-541fa3e20363","Type":"ContainerStarted","Data":"2c9aef12d615f09ef1c9693aa994216148620ab869608eb1acd3ca5028c179e7"}
Sep 18 18:01:24.249343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.249309    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2k9s" event={"ID":"4965f0d2-8952-4a08-b640-541fa3e20363","Type":"ContainerStarted","Data":"3b7014666f9e41d207f6f54ec60217aedfc2c397d3f10d38ef156bad46fc14f7"}
Sep 18 18:01:24.273455 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.273411    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s2k9s" podStartSLOduration=17.604998014 podStartE2EDuration="20.273395339s" totalImagesPullingTime="2.668397325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:01:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:24.272114453 +0000 UTC m=+67.699327208" watchObservedRunningTime="2026-09-18 18:01:24.273395339 +0000 UTC m=+67.700608093"
Sep 18 18:01:24.395343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.395312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:01:24.395507 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.395370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:24.397800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.397774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"image-registry-76756c9684-jxrnf\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") " pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:24.397901 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.397811    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c502493-7814-4874-bac9-96021e28557d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gp99n\" (UID: \"2c502493-7814-4874-bac9-96021e28557d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:01:24.496406 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.496373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:24.496406 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.496409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:24.496613 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.496426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:01:24.496990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.496964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca87bb38-f503-4312-8ea0-ea8941f1ea67-service-ca-bundle\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:24.498768 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.498743    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca87bb38-f503-4312-8ea0-ea8941f1ea67-metrics-certs\") pod \"router-default-598c64547d-8cwhs\" (UID: \"ca87bb38-f503-4312-8ea0-ea8941f1ea67\") " pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:24.498855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.498775    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3558a4-49fc-4e96-986f-c1a7821da11a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-hxhc4\" (UID: \"6a3558a4-49fc-4e96-986f-c1a7821da11a\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:01:24.596897 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.596869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:01:24.597070 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.596909    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:24.599194 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.599164    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/995ff60d-7dd5-491a-aca8-34ce1216a8bc-metrics-tls\") pod \"dns-default-vwp4c\" (UID: \"995ff60d-7dd5-491a-aca8-34ce1216a8bc\") " pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:24.599752 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.599738    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:01:24.603181 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.603160    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qzvvm\""
Sep 18 18:01:24.609009 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.608990    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6a1fbbd-d154-4857-9e94-f4a938082b6c-original-pull-secret\") pod \"global-pull-secret-syncer-8grrr\" (UID: \"d6a1fbbd-d154-4857-9e94-f4a938082b6c\") " pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:01:24.610964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.610950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:24.636424 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.636400    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vzjmc\""
Sep 18 18:01:24.643990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.643968    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"
Sep 18 18:01:24.697747 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.697717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:01:24.700644 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.700609    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:01:24.711027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.711006    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c-metrics-certs\") pod \"network-metrics-daemon-d9bp5\" (UID: \"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c\") " pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:01:24.727731 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.727711    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kk545\""
Sep 18 18:01:24.735991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.735976    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"
Sep 18 18:01:24.740524 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.740509    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4rqgg\""
Sep 18 18:01:24.748693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.748678    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:24.807934 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.807888    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhsq2\""
Sep 18 18:01:24.814273 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.813919    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:24.837890 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.837865    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kxcq\""
Sep 18 18:01:24.843841 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.843815    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8grrr"
Sep 18 18:01:24.845634 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.845614    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9bp5"
Sep 18 18:01:24.900551 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.900523    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76756c9684-jxrnf"]
Sep 18 18:01:24.903570 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:24.903544    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a3834a_cc44_470d_8af5_665d4a6e5463.slice/crio-0ca19b6d4e802be30fc5f05963f54ddfbe171bc133b60e3655a0ae7e0af529cb WatchSource:0}: Error finding container 0ca19b6d4e802be30fc5f05963f54ddfbe171bc133b60e3655a0ae7e0af529cb: Status 404 returned error can't find the container with id 0ca19b6d4e802be30fc5f05963f54ddfbe171bc133b60e3655a0ae7e0af529cb
Sep 18 18:01:24.930133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:24.930108    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n"]
Sep 18 18:01:25.004597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.004569    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4"]
Sep 18 18:01:25.007407 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:25.007381    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3558a4_49fc_4e96_986f_c1a7821da11a.slice/crio-8ca898bffab8d0392db8a5be83b33926df6fe00e6309fcf53a7bec1554ff170e WatchSource:0}: Error finding container 8ca898bffab8d0392db8a5be83b33926df6fe00e6309fcf53a7bec1554ff170e: Status 404 returned error can't find the container with id 8ca898bffab8d0392db8a5be83b33926df6fe00e6309fcf53a7bec1554ff170e
Sep 18 18:01:25.028787 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.028747    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-598c64547d-8cwhs"]
Sep 18 18:01:25.030893 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:25.030871    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca87bb38_f503_4312_8ea0_ea8941f1ea67.slice/crio-e219313daa821bdaf08bc094e702855ed48ef102a574b2d9058cacb0d10ae36d WatchSource:0}: Error finding container e219313daa821bdaf08bc094e702855ed48ef102a574b2d9058cacb0d10ae36d: Status 404 returned error can't find the container with id e219313daa821bdaf08bc094e702855ed48ef102a574b2d9058cacb0d10ae36d
Sep 18 18:01:25.084251 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.084158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vwp4c"]
Sep 18 18:01:25.087762 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:25.087740    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995ff60d_7dd5_491a_aca8_34ce1216a8bc.slice/crio-c7fe0be08a16bf18db093e655b66d24ae226a7ccbac2ecb347bc1340fd345adf WatchSource:0}: Error finding container c7fe0be08a16bf18db093e655b66d24ae226a7ccbac2ecb347bc1340fd345adf: Status 404 returned error can't find the container with id c7fe0be08a16bf18db093e655b66d24ae226a7ccbac2ecb347bc1340fd345adf
Sep 18 18:01:25.132536 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.132468    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8grrr"]
Sep 18 18:01:25.136309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.136266    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d9bp5"]
Sep 18 18:01:25.136309 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:25.136287    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a1fbbd_d154_4857_9e94_f4a938082b6c.slice/crio-569b8c8385a4a9f98fb992dcd6c664a64799b29c42b0c86e0dfda4a9a05e21f0 WatchSource:0}: Error finding container 569b8c8385a4a9f98fb992dcd6c664a64799b29c42b0c86e0dfda4a9a05e21f0: Status 404 returned error can't find the container with id 569b8c8385a4a9f98fb992dcd6c664a64799b29c42b0c86e0dfda4a9a05e21f0
Sep 18 18:01:25.138848 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:25.138828    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c0c8d7_9c65_4a58_94e8_05506ecf9e7c.slice/crio-840b79cff0bdb64b91cc85dbc7c9dd78997e1da9d23deb2a68ad5e0682d806aa WatchSource:0}: Error finding container 840b79cff0bdb64b91cc85dbc7c9dd78997e1da9d23deb2a68ad5e0682d806aa: Status 404 returned error can't find the container with id 840b79cff0bdb64b91cc85dbc7c9dd78997e1da9d23deb2a68ad5e0682d806aa
Sep 18 18:01:25.254845 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.254809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n" event={"ID":"2c502493-7814-4874-bac9-96021e28557d","Type":"ContainerStarted","Data":"aa4bb002c48a6f55e625e57f03aaee57913d1ec163a0cf49859406afec4e32da"}
Sep 18 18:01:25.256103 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.256075    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8grrr" event={"ID":"d6a1fbbd-d154-4857-9e94-f4a938082b6c","Type":"ContainerStarted","Data":"569b8c8385a4a9f98fb992dcd6c664a64799b29c42b0c86e0dfda4a9a05e21f0"}
Sep 18 18:01:25.257250 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.257226    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4" event={"ID":"6a3558a4-49fc-4e96-986f-c1a7821da11a","Type":"ContainerStarted","Data":"8ca898bffab8d0392db8a5be83b33926df6fe00e6309fcf53a7bec1554ff170e"}
Sep 18 18:01:25.258784 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.258762    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76756c9684-jxrnf" event={"ID":"c3a3834a-cc44-470d-8af5-665d4a6e5463","Type":"ContainerStarted","Data":"501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2"}
Sep 18 18:01:25.258866 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.258798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76756c9684-jxrnf" event={"ID":"c3a3834a-cc44-470d-8af5-665d4a6e5463","Type":"ContainerStarted","Data":"0ca19b6d4e802be30fc5f05963f54ddfbe171bc133b60e3655a0ae7e0af529cb"}
Sep 18 18:01:25.258866 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.258842    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:25.259944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.259923    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d9bp5" event={"ID":"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c","Type":"ContainerStarted","Data":"840b79cff0bdb64b91cc85dbc7c9dd78997e1da9d23deb2a68ad5e0682d806aa"}
Sep 18 18:01:25.261075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.261029    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vwp4c" event={"ID":"995ff60d-7dd5-491a-aca8-34ce1216a8bc","Type":"ContainerStarted","Data":"c7fe0be08a16bf18db093e655b66d24ae226a7ccbac2ecb347bc1340fd345adf"}
Sep 18 18:01:25.262575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.262554    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-598c64547d-8cwhs" event={"ID":"ca87bb38-f503-4312-8ea0-ea8941f1ea67","Type":"ContainerStarted","Data":"c2b812204eabc883eb359c5b0745c73803c2db38b1b21d88e8b3163495d2891a"}
Sep 18 18:01:25.262666 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.262592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-598c64547d-8cwhs" event={"ID":"ca87bb38-f503-4312-8ea0-ea8941f1ea67","Type":"ContainerStarted","Data":"e219313daa821bdaf08bc094e702855ed48ef102a574b2d9058cacb0d10ae36d"}
Sep 18 18:01:25.278627 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.278590    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76756c9684-jxrnf" podStartSLOduration=66.27857694 podStartE2EDuration="1m6.27857694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:25.27843711 +0000 UTC m=+68.705649866" watchObservedRunningTime="2026-09-18 18:01:25.27857694 +0000 UTC m=+68.705789694"
Sep 18 18:01:25.297741 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.297689    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-598c64547d-8cwhs" podStartSLOduration=59.297673368 podStartE2EDuration="59.297673368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:25.296643608 +0000 UTC m=+68.723856362" watchObservedRunningTime="2026-09-18 18:01:25.297673368 +0000 UTC m=+68.724886116"
Sep 18 18:01:25.749829 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.749779    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:25.753548 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.753515    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:25.915259 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:25.914834    2582 scope.go:117] "RemoveContainer" containerID="a96bbe4cdf5e19d51ec92ef391902c687fb112306667e1e4dd5488ad9931877f"
Sep 18 18:01:26.213009 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.212963    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:01:26.215873 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.215843    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2b950ab-bdf2-4d50-8467-d4d2e762af1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-g47n6\" (UID: \"b2b950ab-bdf2-4d50-8467-d4d2e762af1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:01:26.275240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.275166    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:01:26.275668 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.275471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" event={"ID":"c19150e6-187a-4e99-907e-b584d8b3aa55","Type":"ContainerStarted","Data":"a65f843d46038a9233b71969a4d19d6487055b372b3c7ffd87e008d53a52cff7"}
Sep 18 18:01:26.276333 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.276309    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:26.276461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.276356    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:01:26.277357 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.277336    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-598c64547d-8cwhs"
Sep 18 18:01:26.316364 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.316304    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm" podStartSLOduration=50.419457629 podStartE2EDuration="1m0.316283581s" totalImagesPullingTime="9.896825952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:26.295330316 +0000 UTC m=+69.722543071" watchObservedRunningTime="2026-09-18 18:01:26.316283581 +0000 UTC m=+69.743496352"
Sep 18 18:01:26.455179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.455151    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7gpcf\""
Sep 18 18:01:26.463309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.463281    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"
Sep 18 18:01:26.837607 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:26.837526    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-w8lzm"
Sep 18 18:01:30.176234 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.173019    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-g47n6"]
Sep 18 18:01:30.183882 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:30.183851    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b950ab_bdf2_4d50_8467_d4d2e762af1f.slice/crio-83571f87bd6b68494a49e1285c17f0083a493a16fbe20324d1c836fa359a7e8d WatchSource:0}: Error finding container 83571f87bd6b68494a49e1285c17f0083a493a16fbe20324d1c836fa359a7e8d: Status 404 returned error can't find the container with id 83571f87bd6b68494a49e1285c17f0083a493a16fbe20324d1c836fa359a7e8d
Sep 18 18:01:30.295542 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.295502    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6" event={"ID":"b2b950ab-bdf2-4d50-8467-d4d2e762af1f","Type":"ContainerStarted","Data":"83571f87bd6b68494a49e1285c17f0083a493a16fbe20324d1c836fa359a7e8d"}
Sep 18 18:01:30.298501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.297527    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d9bp5" event={"ID":"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c","Type":"ContainerStarted","Data":"26f8a2f7ef39ec5801285de40460aefcfcb900bd142e1f2374a1e0f73775ff7d"}
Sep 18 18:01:30.298501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.297610    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d9bp5" event={"ID":"d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c","Type":"ContainerStarted","Data":"efd19b4aaa69807bc167962d448cb3f57d21e91fd382c48ecd3ca0d3c17c6aed"}
Sep 18 18:01:30.301558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.301424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vwp4c" event={"ID":"995ff60d-7dd5-491a-aca8-34ce1216a8bc","Type":"ContainerStarted","Data":"67e8394946ab1673848a97939fc0110ffcc626cab15c1d2c113a44d9ac734fc3"}
Sep 18 18:01:30.301558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.301472    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vwp4c" event={"ID":"995ff60d-7dd5-491a-aca8-34ce1216a8bc","Type":"ContainerStarted","Data":"01cc2e6f7b351debb9abce3ecf7cba60fada697a39ca9d1e8f9258dc89c2999e"}
Sep 18 18:01:30.301558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.301514    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:30.303996 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.303938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n" event={"ID":"2c502493-7814-4874-bac9-96021e28557d","Type":"ContainerStarted","Data":"296910832b20b84cbc19225550133cc7b447ae4cb729e99e7ab70044c5f06ba4"}
Sep 18 18:01:30.303996 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.303991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n" event={"ID":"2c502493-7814-4874-bac9-96021e28557d","Type":"ContainerStarted","Data":"aa95206603ebc0e22b1dba455b6beaac50837990573032bcfff094fff0b981c9"}
Sep 18 18:01:30.305887 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.305848    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8grrr" event={"ID":"d6a1fbbd-d154-4857-9e94-f4a938082b6c","Type":"ContainerStarted","Data":"7af5637b853bdf3fc19ef4c3c7e0b6c2e906571f9c0c48be523bc0d85ce72636"}
Sep 18 18:01:30.308087 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.308064    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4" event={"ID":"6a3558a4-49fc-4e96-986f-c1a7821da11a","Type":"ContainerStarted","Data":"b9a37687bb35ae613ccbbc0698f101d7103155e988f78d0d4582543a481c7321"}
Sep 18 18:01:30.319994 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.319942    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d9bp5" podStartSLOduration=66.574560445 podStartE2EDuration="1m11.31992546s" totalImagesPullingTime="4.745365015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:30.319181663 +0000 UTC m=+73.746394414" watchObservedRunningTime="2026-09-18 18:01:30.31992546 +0000 UTC m=+73.747138230"
Sep 18 18:01:30.336442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.336247    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gp99n" podStartSLOduration=59.429407617 podStartE2EDuration="1m4.336231879s" totalImagesPullingTime="4.906824262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:30.336148453 +0000 UTC m=+73.763361206" watchObservedRunningTime="2026-09-18 18:01:30.336231879 +0000 UTC m=+73.763444634"
Sep 18 18:01:30.359078 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.358326    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vwp4c" podStartSLOduration=33.562131763 podStartE2EDuration="38.358307624s" totalImagesPullingTime="4.796175861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:30.354126157 +0000 UTC m=+73.781338913" watchObservedRunningTime="2026-09-18 18:01:30.358307624 +0000 UTC m=+73.785520379"
Sep 18 18:01:30.372040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.371960    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8grrr" podStartSLOduration=66.605290224 podStartE2EDuration="1m11.371946658s" totalImagesPullingTime="4.766656434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:30.370480108 +0000 UTC m=+73.797692873" watchObservedRunningTime="2026-09-18 18:01:30.371946658 +0000 UTC m=+73.799159411"
Sep 18 18:01:30.396181 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:30.396130    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-hxhc4" podStartSLOduration=59.518788941 podStartE2EDuration="1m4.396116415s" totalImagesPullingTime="4.877327474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:30.393512332 +0000 UTC m=+73.820725085" watchObservedRunningTime="2026-09-18 18:01:30.396116415 +0000 UTC m=+73.823329168"
Sep 18 18:01:32.318625 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:32.318592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6" event={"ID":"b2b950ab-bdf2-4d50-8467-d4d2e762af1f","Type":"ContainerStarted","Data":"9b843592fd7be7d068a30ab19a89b6371899cbb2bb338f6f5f765ef64ed3ba35"}
Sep 18 18:01:32.335385 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:32.335335    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-g47n6" podStartSLOduration=36.743401401 podStartE2EDuration="38.335322181s" totalImagesPullingTime="1.59192078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:32.334516764 +0000 UTC m=+75.761729517" watchObservedRunningTime="2026-09-18 18:01:32.335322181 +0000 UTC m=+75.762534935"
Sep 18 18:01:35.174929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.174900    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xp6p5"
Sep 18 18:01:35.604224 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.604187    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"]
Sep 18 18:01:35.627213 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.627186    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"]
Sep 18 18:01:35.627355 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.627287    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.629852 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.629810    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 18:01:35.629852 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.629843    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 18:01:35.630040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.629891    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w9b4h\""
Sep 18 18:01:35.630040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.630018    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 18:01:35.695171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.695136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.695171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.695177    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwssw\" (UniqueName: \"kubernetes.io/projected/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-kube-api-access-fwssw\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.695357 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.695258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.695357 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.695302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.795833 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.795807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.795964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.795847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.795964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.795871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.795964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.795932    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwssw\" (UniqueName: \"kubernetes.io/projected/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-kube-api-access-fwssw\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.796146 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:35.796129    2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 18 18:01:35.796228 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:35.796216    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-tls podName:b99311ed-0547-4c7c-a417-ba5ec0a32d8f nodeName:}" failed. No retries permitted until 2026-09-18 18:01:36.296195287 +0000 UTC m=+79.723408032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-jx4j6" (UID: "b99311ed-0547-4c7c-a417-ba5ec0a32d8f") : secret "prometheus-operator-tls" not found
Sep 18 18:01:35.796667 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.796650    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.798421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.798401    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:35.807031 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:35.807004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwssw\" (UniqueName: \"kubernetes.io/projected/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-kube-api-access-fwssw\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:36.299558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:36.299525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:36.302098 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:36.302074    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b99311ed-0547-4c7c-a417-ba5ec0a32d8f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-jx4j6\" (UID: \"b99311ed-0547-4c7c-a417-ba5ec0a32d8f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:36.535843 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:36.535805    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"
Sep 18 18:01:36.803894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:36.803862    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6"]
Sep 18 18:01:36.806724 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:36.806699    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99311ed_0547_4c7c_a417_ba5ec0a32d8f.slice/crio-22ababa5b14615215509a8057294d0508c507a512e419beb0a1b2d0fdcb2ee86 WatchSource:0}: Error finding container 22ababa5b14615215509a8057294d0508c507a512e419beb0a1b2d0fdcb2ee86: Status 404 returned error can't find the container with id 22ababa5b14615215509a8057294d0508c507a512e419beb0a1b2d0fdcb2ee86
Sep 18 18:01:37.337792 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:37.337753    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6" event={"ID":"b99311ed-0547-4c7c-a417-ba5ec0a32d8f","Type":"ContainerStarted","Data":"22ababa5b14615215509a8057294d0508c507a512e419beb0a1b2d0fdcb2ee86"}
Sep 18 18:01:39.346426 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:39.346383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6" event={"ID":"b99311ed-0547-4c7c-a417-ba5ec0a32d8f","Type":"ContainerStarted","Data":"83c522eaa96562d027a260bfa68dfa40f5b63cfc922d7d961408056adeeeb63b"}
Sep 18 18:01:39.346795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:39.346439    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6" event={"ID":"b99311ed-0547-4c7c-a417-ba5ec0a32d8f","Type":"ContainerStarted","Data":"2119958896b6db657548dda627b6694f781cc5b495f22a832a803abc19b61fda"}
Sep 18 18:01:39.370254 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:39.370206    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-jx4j6" podStartSLOduration=2.832793912 podStartE2EDuration="4.370191482s" totalImagesPullingTime="1.53739757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:39.36903221 +0000 UTC m=+82.796244964" watchObservedRunningTime="2026-09-18 18:01:39.370191482 +0000 UTC m=+82.797404263"
Sep 18 18:01:40.315247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:40.315215    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vwp4c"
Sep 18 18:01:41.016127 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.016092    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-rcc59"]
Sep 18 18:01:41.019492 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.019468    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.023439 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.023406    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 18 18:01:41.023439 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.023428    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4t7mk\""
Sep 18 18:01:41.023600 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.023428    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 18 18:01:41.023688 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.023675    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 18 18:01:41.043629 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.043598    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-rcc59"]
Sep 18 18:01:41.045006 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.044987    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mkphf"]
Sep 18 18:01:41.047125 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.047107    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.049709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.049684    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 18:01:41.049806 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.049759    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 18:01:41.049855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.049759    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 18:01:41.049903 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.049870    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9xvzz\""
Sep 18 18:01:41.136501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136466    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7f4117fc-34f7-4742-96c6-88a1e481a63d-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.136501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.136693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136584    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.136693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.136693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-sys\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-tls\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136710    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-textfile\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrlg9\" (UniqueName: \"kubernetes.io/projected/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-api-access-xrlg9\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.136847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f4117fc-34f7-4742-96c6-88a1e481a63d-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.136847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136810    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwpwm\" (UniqueName: \"kubernetes.io/projected/8a6d5da1-0016-409e-a3a8-203938e69149-kube-api-access-mwpwm\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-root\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6d5da1-0016-409e-a3a8-203938e69149-metrics-client-ca\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136879    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-wtmp\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.136986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.136908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-accelerators-collector-config\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238283 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.238465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238322    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.238465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-sys\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-tls\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-textfile\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrlg9\" (UniqueName: \"kubernetes.io/projected/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-api-access-xrlg9\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.238724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238517    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f4117fc-34f7-4742-96c6-88a1e481a63d-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.238724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238571    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-sys\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwpwm\" (UniqueName: \"kubernetes.io/projected/8a6d5da1-0016-409e-a3a8-203938e69149-kube-api-access-mwpwm\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-root\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.238724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6d5da1-0016-409e-a3a8-203938e69149-metrics-client-ca\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.239017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-wtmp\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.239017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-accelerators-collector-config\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.239017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238784    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-textfile\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.239017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7f4117fc-34f7-4742-96c6-88a1e481a63d-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.239017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.239017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.238886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-root\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.239529 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.239042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-wtmp\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.239529 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.239411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7f4117fc-34f7-4742-96c6-88a1e481a63d-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.239529 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.239494    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.239686 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.239587    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a6d5da1-0016-409e-a3a8-203938e69149-metrics-client-ca\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.240074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.240033    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-accelerators-collector-config\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.240206 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.240185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f4117fc-34f7-4742-96c6-88a1e481a63d-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.241141 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.241112    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-tls\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.241237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.241215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a6d5da1-0016-409e-a3a8-203938e69149-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.241276 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.241256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.241624 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.241603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.248492 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.248469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrlg9\" (UniqueName: \"kubernetes.io/projected/7f4117fc-34f7-4742-96c6-88a1e481a63d-kube-api-access-xrlg9\") pod \"kube-state-metrics-84c885658c-rcc59\" (UID: \"7f4117fc-34f7-4742-96c6-88a1e481a63d\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.250318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.250295    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwpwm\" (UniqueName: \"kubernetes.io/projected/8a6d5da1-0016-409e-a3a8-203938e69149-kube-api-access-mwpwm\") pod \"node-exporter-mkphf\" (UID: \"8a6d5da1-0016-409e-a3a8-203938e69149\") " pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.329745 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.329711    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59"
Sep 18 18:01:41.358361 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.358334    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mkphf"
Sep 18 18:01:41.369316 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:41.369283    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6d5da1_0016_409e_a3a8_203938e69149.slice/crio-ab92ea8c17cdc261110c59373dd04d018edeb57aa22a283a3fe407ff52cadc1a WatchSource:0}: Error finding container ab92ea8c17cdc261110c59373dd04d018edeb57aa22a283a3fe407ff52cadc1a: Status 404 returned error can't find the container with id ab92ea8c17cdc261110c59373dd04d018edeb57aa22a283a3fe407ff52cadc1a
Sep 18 18:01:41.602195 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:41.602113    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-rcc59"]
Sep 18 18:01:41.605392 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:41.605364    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4117fc_34f7_4742_96c6_88a1e481a63d.slice/crio-6916748ebf9384ef9b158d896867d571bb99fa4287893aafb03c75460e228696 WatchSource:0}: Error finding container 6916748ebf9384ef9b158d896867d571bb99fa4287893aafb03c75460e228696: Status 404 returned error can't find the container with id 6916748ebf9384ef9b158d896867d571bb99fa4287893aafb03c75460e228696
Sep 18 18:01:42.360030 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:42.359991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkphf" event={"ID":"8a6d5da1-0016-409e-a3a8-203938e69149","Type":"ContainerStarted","Data":"ab92ea8c17cdc261110c59373dd04d018edeb57aa22a283a3fe407ff52cadc1a"}
Sep 18 18:01:42.361293 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:42.361263    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59" event={"ID":"7f4117fc-34f7-4742-96c6-88a1e481a63d","Type":"ContainerStarted","Data":"6916748ebf9384ef9b158d896867d571bb99fa4287893aafb03c75460e228696"}
Sep 18 18:01:43.368550 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:43.368514    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59" event={"ID":"7f4117fc-34f7-4742-96c6-88a1e481a63d","Type":"ContainerStarted","Data":"98d369ad7974d552b5e6d4b4cd14abedc625c35c4694856d8301e2e11c51448a"}
Sep 18 18:01:43.368941 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:43.368588    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59" event={"ID":"7f4117fc-34f7-4742-96c6-88a1e481a63d","Type":"ContainerStarted","Data":"6826f3ac3c44b7c113995dc1ef4850253d7f72b933f44bebe39b874becdeb145"}
Sep 18 18:01:43.370673 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:43.370646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkphf" event={"ID":"8a6d5da1-0016-409e-a3a8-203938e69149","Type":"ContainerStarted","Data":"548f32882b530d853b462018277f836e3e596772f091bb426fafe15b8c11c424"}
Sep 18 18:01:44.376226 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:44.376191    2582 generic.go:356] "Generic (PLEG): container finished" podID="8a6d5da1-0016-409e-a3a8-203938e69149" containerID="548f32882b530d853b462018277f836e3e596772f091bb426fafe15b8c11c424" exitCode=0
Sep 18 18:01:44.376629 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:44.376277    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkphf" event={"ID":"8a6d5da1-0016-409e-a3a8-203938e69149","Type":"ContainerDied","Data":"548f32882b530d853b462018277f836e3e596772f091bb426fafe15b8c11c424"}
Sep 18 18:01:44.378595 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:44.378569    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59" event={"ID":"7f4117fc-34f7-4742-96c6-88a1e481a63d","Type":"ContainerStarted","Data":"fb63ee46b2f4d822906dd293808ac5eb6881bd41d65879aa98175c5bd1339479"}
Sep 18 18:01:44.417691 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:44.417637    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-rcc59" podStartSLOduration=2.827842063 podStartE2EDuration="4.417618341s" totalImagesPullingTime="1.589776278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:44.415542017 +0000 UTC m=+87.842754772" watchObservedRunningTime="2026-09-18 18:01:44.417618341 +0000 UTC m=+87.844831097"
Sep 18 18:01:45.385037 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.384997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkphf" event={"ID":"8a6d5da1-0016-409e-a3a8-203938e69149","Type":"ContainerStarted","Data":"a367e200c8d9196f43267b5a0c6b4bfc2d0bb489cd1e42915a93869dc71f55d4"}
Sep 18 18:01:45.385434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.385068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mkphf" event={"ID":"8a6d5da1-0016-409e-a3a8-203938e69149","Type":"ContainerStarted","Data":"fbdb915f553f2da888731d9916f4f2afb219d22f4177d690296b83a776b703a9"}
Sep 18 18:01:45.411489 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.411438    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mkphf" podStartSLOduration=2.490084468 podStartE2EDuration="4.411424474s" totalImagesPullingTime="1.823708007s" totalInitContainerRuntime="97.631999ms" isStatefulPod=true podCreationTimestamp="2026-09-18 18:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:45.411362807 +0000 UTC m=+88.838575561" watchObservedRunningTime="2026-09-18 18:01:45.411424474 +0000 UTC m=+88.838637227"
Sep 18 18:01:45.754668 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.754592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"]
Sep 18 18:01:45.756931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.756903    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:45.759929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.759906    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gnvcw\""
Sep 18 18:01:45.760163 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.760149    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 18 18:01:45.771902 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.771880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"]
Sep 18 18:01:45.809472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.809442    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-788d44d748-44r7k"]
Sep 18 18:01:45.811645 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.811627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.814327 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.814299    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 18:01:45.815990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.815971    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 18:01:45.816139 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.816119    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 18:01:45.816711 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.816693    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 18:01:45.816846 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.816822    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 18:01:45.816979 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.816909    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 18:01:45.817308 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.817286    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ffzz9\""
Sep 18 18:01:45.817402 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.817320    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 18:01:45.823111 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.823091    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 18:01:45.827431 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.827406    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-788d44d748-44r7k"]
Sep 18 18:01:45.880599 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.880569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53153c7a-a902-474b-b7f7-2af4ef8461f6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:45.880723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.880612    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ghz\" (UniqueName: \"kubernetes.io/projected/53153c7a-a902-474b-b7f7-2af4ef8461f6-kube-api-access-f8ghz\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:45.981331 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981300    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-config\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ghz\" (UniqueName: \"kubernetes.io/projected/53153c7a-a902-474b-b7f7-2af4ef8461f6-kube-api-access-f8ghz\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:45.981434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-trusted-ca-bundle\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981516 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-oauth-config\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981516 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-oauth-serving-cert\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981516 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981493    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zffk5\" (UniqueName: \"kubernetes.io/projected/3e84a46f-0d53-44ae-9db2-decbcc1094f0-kube-api-access-zffk5\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981640 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-service-ca\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981640 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53153c7a-a902-474b-b7f7-2af4ef8461f6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:45.981640 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.981592    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-serving-cert\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:45.981729 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:45.981665    2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 18 18:01:45.981729 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:01:45.981717    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53153c7a-a902-474b-b7f7-2af4ef8461f6-monitoring-plugin-cert podName:53153c7a-a902-474b-b7f7-2af4ef8461f6 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:46.481703428 +0000 UTC m=+89.908916160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/53153c7a-a902-474b-b7f7-2af4ef8461f6-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-d8n5h" (UID: "53153c7a-a902-474b-b7f7-2af4ef8461f6") : secret "monitoring-plugin-cert" not found
Sep 18 18:01:45.990813 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:45.990792    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ghz\" (UniqueName: \"kubernetes.io/projected/53153c7a-a902-474b-b7f7-2af4ef8461f6-kube-api-access-f8ghz\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:46.082056 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082024    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-config\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-trusted-ca-bundle\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-oauth-config\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-oauth-serving-cert\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082160    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zffk5\" (UniqueName: \"kubernetes.io/projected/3e84a46f-0d53-44ae-9db2-decbcc1094f0-kube-api-access-zffk5\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-service-ca\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082455 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-serving-cert\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082980 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082895    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-config\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.082980 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.082922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-service-ca\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.083184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.083116    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-oauth-serving-cert\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.083241 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.083220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-trusted-ca-bundle\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.084994 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.084970    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-oauth-config\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.085132 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.085109    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-serving-cert\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.091980 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.091955    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zffk5\" (UniqueName: \"kubernetes.io/projected/3e84a46f-0d53-44ae-9db2-decbcc1094f0-kube-api-access-zffk5\") pod \"console-788d44d748-44r7k\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") " pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.121035 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.121007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:01:46.260158 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.260121    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5bf777bb65-fphk8"]
Sep 18 18:01:46.262980 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.262964    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.267975 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.267955    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 18:01:46.267975 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.267967    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 18:01:46.268180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.267966    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 18:01:46.268180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.267967    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-l5mtj\""
Sep 18 18:01:46.268180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.267967    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 18:01:46.268372 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.268355    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 18:01:46.273759 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.273740    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 18:01:46.281141 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.281117    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bf777bb65-fphk8"]
Sep 18 18:01:46.281549 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.281529    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:01:46.386030 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.385932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k86f\" (UniqueName: \"kubernetes.io/projected/46e4eef9-a010-445e-a81b-5f7489785a31-kube-api-access-8k86f\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386030 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.385998    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-serving-certs-ca-bundle\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386030 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.386024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-federate-client-tls\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.386132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.386206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-secret-telemeter-client\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.386237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.386314    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-metrics-client-ca\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.386586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.386408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-telemeter-client-tls\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.416626 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.416599    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-788d44d748-44r7k"]
Sep 18 18:01:46.419455 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:46.419419    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e84a46f_0d53_44ae_9db2_decbcc1094f0.slice/crio-ddcf33bf296e1e904262ad67280888a99361bbb7cb14ce3800beb7cd0044b65e WatchSource:0}: Error finding container ddcf33bf296e1e904262ad67280888a99361bbb7cb14ce3800beb7cd0044b65e: Status 404 returned error can't find the container with id ddcf33bf296e1e904262ad67280888a99361bbb7cb14ce3800beb7cd0044b65e
Sep 18 18:01:46.486964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.486933    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53153c7a-a902-474b-b7f7-2af4ef8461f6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:46.486964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.486966    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-telemeter-client-tls\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.486987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k86f\" (UniqueName: \"kubernetes.io/projected/46e4eef9-a010-445e-a81b-5f7489785a31-kube-api-access-8k86f\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.487023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-serving-certs-ca-bundle\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.487072    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-federate-client-tls\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.487135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.487190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-secret-telemeter-client\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.487221    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.487544 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.487322    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-metrics-client-ca\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.488141 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.488091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-metrics-client-ca\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.488612 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.488583    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.488730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.488698    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e4eef9-a010-445e-a81b-5f7489785a31-serving-certs-ca-bundle\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.489972 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.489948    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.490142 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.490122    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-federate-client-tls\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.490205 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.490169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-secret-telemeter-client\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.490255 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.490210    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/53153c7a-a902-474b-b7f7-2af4ef8461f6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-d8n5h\" (UID: \"53153c7a-a902-474b-b7f7-2af4ef8461f6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:46.490255 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.490242    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/46e4eef9-a010-445e-a81b-5f7489785a31-telemeter-client-tls\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.495572 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.495555    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k86f\" (UniqueName: \"kubernetes.io/projected/46e4eef9-a010-445e-a81b-5f7489785a31-kube-api-access-8k86f\") pod \"telemeter-client-5bf777bb65-fphk8\" (UID: \"46e4eef9-a010-445e-a81b-5f7489785a31\") " pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.573639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.573606    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8"
Sep 18 18:01:46.666550 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.666451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:46.853170 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.852266    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bf777bb65-fphk8"]
Sep 18 18:01:46.941078 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:46.940970    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"]
Sep 18 18:01:46.945298 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:01:46.945265    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53153c7a_a902_474b_b7f7_2af4ef8461f6.slice/crio-98e651af8f6601218bfff92e14b0b535da617cfae25e8652f35aa34741eaa11b WatchSource:0}: Error finding container 98e651af8f6601218bfff92e14b0b535da617cfae25e8652f35aa34741eaa11b: Status 404 returned error can't find the container with id 98e651af8f6601218bfff92e14b0b535da617cfae25e8652f35aa34741eaa11b
Sep 18 18:01:47.401862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:47.401767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h" event={"ID":"53153c7a-a902-474b-b7f7-2af4ef8461f6","Type":"ContainerStarted","Data":"98e651af8f6601218bfff92e14b0b535da617cfae25e8652f35aa34741eaa11b"}
Sep 18 18:01:47.406529 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:47.406494    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788d44d748-44r7k" event={"ID":"3e84a46f-0d53-44ae-9db2-decbcc1094f0","Type":"ContainerStarted","Data":"ddcf33bf296e1e904262ad67280888a99361bbb7cb14ce3800beb7cd0044b65e"}
Sep 18 18:01:47.408188 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:47.408095    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8" event={"ID":"46e4eef9-a010-445e-a81b-5f7489785a31","Type":"ContainerStarted","Data":"67baabb21a6c375730ec1ab64aa77fe1d18be3b7b662c8399f816e63b5a37a38"}
Sep 18 18:01:50.030205 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:50.030170    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76756c9684-jxrnf"]
Sep 18 18:01:52.434988 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.434945    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h" event={"ID":"53153c7a-a902-474b-b7f7-2af4ef8461f6","Type":"ContainerStarted","Data":"0da9abf4f01c648662e50b61002b1f9d205d6258c13b2aba8588410f1c87e2a2"}
Sep 18 18:01:52.436097 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.436071    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:52.440683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.440618    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788d44d748-44r7k" event={"ID":"3e84a46f-0d53-44ae-9db2-decbcc1094f0","Type":"ContainerStarted","Data":"48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590"}
Sep 18 18:01:52.442405 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.442381    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h"
Sep 18 18:01:52.442511 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.442416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8" event={"ID":"46e4eef9-a010-445e-a81b-5f7489785a31","Type":"ContainerStarted","Data":"38a8a47a69231c350bc7bf2a8533c56dae79c99f14ac4d3f6224ba8fa6166f05"}
Sep 18 18:01:52.454260 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.454123    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-d8n5h" podStartSLOduration=2.874474937 podStartE2EDuration="7.454102307s" totalImagesPullingTime="4.57962737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:52.451622304 +0000 UTC m=+95.878835059" watchObservedRunningTime="2026-09-18 18:01:52.454102307 +0000 UTC m=+95.881315063"
Sep 18 18:01:52.475510 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:52.475455    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-788d44d748-44r7k" podStartSLOduration=2.3334191779999998 podStartE2EDuration="7.475433498s" totalImagesPullingTime="5.14201432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:52.472833654 +0000 UTC m=+95.900046421" watchObservedRunningTime="2026-09-18 18:01:52.475433498 +0000 UTC m=+95.902646252"
Sep 18 18:01:53.448977 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:53.448933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8" event={"ID":"46e4eef9-a010-445e-a81b-5f7489785a31","Type":"ContainerStarted","Data":"03590628b277ba611bcd2f978e423c3f9f5f8453f87d219934bfdfe7b6084cbe"}
Sep 18 18:01:53.449411 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:53.449004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8" event={"ID":"46e4eef9-a010-445e-a81b-5f7489785a31","Type":"ContainerStarted","Data":"46e0297a8210fefded67f061ab6518300e9abe841cc9d4bb1dabad8ba55bf2ff"}
Sep 18 18:01:53.472477 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:53.472430    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5bf777bb65-fphk8" podStartSLOduration=1.782527872 podStartE2EDuration="7.472409826s" totalImagesPullingTime="5.689881954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:53.469632054 +0000 UTC m=+96.896844807" watchObservedRunningTime="2026-09-18 18:01:53.472409826 +0000 UTC m=+96.899622612"
Sep 18 18:01:53.947927 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:53.947890    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-788d44d748-44r7k"]
Sep 18 18:01:56.122187 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:01:56.122150    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:02:14.533074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:14.533025    2582 generic.go:356] "Generic (PLEG): container finished" podID="4b17ce39-8f2d-4496-94d3-b62b3c984857" containerID="4d11d91a77b6d500b35381281df13c63a0c0d5b6ced3159ca0bdb20eb9c4cf75" exitCode=0
Sep 18 18:02:14.533584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:14.533104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl" event={"ID":"4b17ce39-8f2d-4496-94d3-b62b3c984857","Type":"ContainerDied","Data":"4d11d91a77b6d500b35381281df13c63a0c0d5b6ced3159ca0bdb20eb9c4cf75"}
Sep 18 18:02:14.533584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:14.533506    2582 scope.go:117] "RemoveContainer" containerID="4d11d91a77b6d500b35381281df13c63a0c0d5b6ced3159ca0bdb20eb9c4cf75"
Sep 18 18:02:15.051559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.051516    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76756c9684-jxrnf" podUID="c3a3834a-cc44-470d-8af5-665d4a6e5463" containerName="registry" containerID="cri-o://501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2" gracePeriod=30
Sep 18 18:02:15.299655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.299628    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:02:15.338474 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338392    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj4p9\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-kube-api-access-jj4p9\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338474 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338437    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-bound-sa-token\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338485    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-installation-pull-secrets\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338534    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-trusted-ca\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338570    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-image-registry-private-configuration\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338606    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-certificates\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.338921 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338695    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3a3834a-cc44-470d-8af5-665d4a6e5463-ca-trust-extracted\") pod \"c3a3834a-cc44-470d-8af5-665d4a6e5463\" (UID: \"c3a3834a-cc44-470d-8af5-665d4a6e5463\") "
Sep 18 18:02:15.339005 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.338975    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:15.340084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.339629    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:15.341327 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.341298    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-kube-api-access-jj4p9" (OuterVolumeSpecName: "kube-api-access-jj4p9") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "kube-api-access-jj4p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:02:15.341579 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.341559    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:02:15.341660 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.341594    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:15.341660 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.341647    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:15.341777 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.341754    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:02:15.348028 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.347997    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a3834a-cc44-470d-8af5-665d4a6e5463-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c3a3834a-cc44-470d-8af5-665d4a6e5463" (UID: "c3a3834a-cc44-470d-8af5-665d4a6e5463"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:02:15.439827 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439795    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-image-registry-private-configuration\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.439827 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439822    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.439827 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439832    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-registry-certificates\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.440074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439842    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3a3834a-cc44-470d-8af5-665d4a6e5463-ca-trust-extracted\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.440074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439851    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj4p9\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-kube-api-access-jj4p9\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.440074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439859    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3a3834a-cc44-470d-8af5-665d4a6e5463-bound-sa-token\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.440074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439867    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3a3834a-cc44-470d-8af5-665d4a6e5463-installation-pull-secrets\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.440074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.439875    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3a3834a-cc44-470d-8af5-665d4a6e5463-trusted-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:15.539424 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.539385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-tm2tl" event={"ID":"4b17ce39-8f2d-4496-94d3-b62b3c984857","Type":"ContainerStarted","Data":"c5bb8b8c9ee6ec28de7c7b817ef730c3e54ddd83deccebbdcbcb7928396f53d9"}
Sep 18 18:02:15.540651 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.540629    2582 generic.go:356] "Generic (PLEG): container finished" podID="c3a3834a-cc44-470d-8af5-665d4a6e5463" containerID="501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2" exitCode=0
Sep 18 18:02:15.540759 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.540676    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76756c9684-jxrnf"
Sep 18 18:02:15.540759 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.540682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76756c9684-jxrnf" event={"ID":"c3a3834a-cc44-470d-8af5-665d4a6e5463","Type":"ContainerDied","Data":"501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2"}
Sep 18 18:02:15.540759 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.540712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76756c9684-jxrnf" event={"ID":"c3a3834a-cc44-470d-8af5-665d4a6e5463","Type":"ContainerDied","Data":"0ca19b6d4e802be30fc5f05963f54ddfbe171bc133b60e3655a0ae7e0af529cb"}
Sep 18 18:02:15.540759 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.540732    2582 scope.go:117] "RemoveContainer" containerID="501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2"
Sep 18 18:02:15.549639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.549622    2582 scope.go:117] "RemoveContainer" containerID="501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2"
Sep 18 18:02:15.549911 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:02:15.549894    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2\": container with ID starting with 501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2 not found: ID does not exist" containerID="501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2"
Sep 18 18:02:15.549969 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.549919    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2"} err="failed to get container status \"501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2\": rpc error: code = NotFound desc = could not find container \"501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2\": container with ID starting with 501e2e5a96d1a4ad43be59e653e191422d316dd2e8bf354b8f48cf519cade2e2 not found: ID does not exist"
Sep 18 18:02:15.570676 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.570652    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76756c9684-jxrnf"]
Sep 18 18:02:15.576593 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:15.576572    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76756c9684-jxrnf"]
Sep 18 18:02:16.919032 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:16.918997    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a3834a-cc44-470d-8af5-665d4a6e5463" path="/var/lib/kubelet/pods/c3a3834a-cc44-470d-8af5-665d4a6e5463/volumes"
Sep 18 18:02:19.475733 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.475685    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-788d44d748-44r7k" podUID="3e84a46f-0d53-44ae-9db2-decbcc1094f0" containerName="console" containerID="cri-o://48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590" gracePeriod=15
Sep 18 18:02:19.558190 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.558157    2582 generic.go:356] "Generic (PLEG): container finished" podID="38dfae07-7770-480a-941b-473232de5af7" containerID="9e9d57bf42f811ef902b22ccfbeb090ad6161940104c886e41dfa5c766f0be7a" exitCode=0
Sep 18 18:02:19.558305 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.558244    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h" event={"ID":"38dfae07-7770-480a-941b-473232de5af7","Type":"ContainerDied","Data":"9e9d57bf42f811ef902b22ccfbeb090ad6161940104c886e41dfa5c766f0be7a"}
Sep 18 18:02:19.558623 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.558605    2582 scope.go:117] "RemoveContainer" containerID="9e9d57bf42f811ef902b22ccfbeb090ad6161940104c886e41dfa5c766f0be7a"
Sep 18 18:02:19.759407 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.759382    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:02:19.876926 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.876889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-serving-cert\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.876938    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-oauth-serving-cert\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.876983    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zffk5\" (UniqueName: \"kubernetes.io/projected/3e84a46f-0d53-44ae-9db2-decbcc1094f0-kube-api-access-zffk5\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877024    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-config\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877076    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-service-ca\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877132    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-trusted-ca-bundle\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877408 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877162    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-oauth-config\") pod \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\" (UID: \"3e84a46f-0d53-44ae-9db2-decbcc1094f0\") "
Sep 18 18:02:19.877462 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877432    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:19.877521 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877487    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-config" (OuterVolumeSpecName: "console-config") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:19.877647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877621    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:19.877972 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.877952    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:19.879559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.879532    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:19.879559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.879536    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e84a46f-0d53-44ae-9db2-decbcc1094f0-kube-api-access-zffk5" (OuterVolumeSpecName: "kube-api-access-zffk5") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "kube-api-access-zffk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:02:19.879705 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.879573    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e84a46f-0d53-44ae-9db2-decbcc1094f0" (UID: "3e84a46f-0d53-44ae-9db2-decbcc1094f0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:19.978683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978654    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-trusted-ca-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:19.978683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978682    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-oauth-config\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:19.978849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978691    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-serving-cert\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:19.978849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978700    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-oauth-serving-cert\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:19.978849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978708    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zffk5\" (UniqueName: \"kubernetes.io/projected/3e84a46f-0d53-44ae-9db2-decbcc1094f0-kube-api-access-zffk5\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:19.978849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978718    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-console-config\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:19.978849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:19.978728    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e84a46f-0d53-44ae-9db2-decbcc1094f0-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:02:20.569445 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.565434    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nht6h" event={"ID":"38dfae07-7770-480a-941b-473232de5af7","Type":"ContainerStarted","Data":"c4853f6f58ad7cc0fb17de8a6f1701da4ab81113845bff99ac8c7e5eaf68b159"}
Sep 18 18:02:20.571227 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.571202    2582 generic.go:356] "Generic (PLEG): container finished" podID="3e84a46f-0d53-44ae-9db2-decbcc1094f0" containerID="48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590" exitCode=0
Sep 18 18:02:20.571350 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.571263    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788d44d748-44r7k"
Sep 18 18:02:20.571350 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.571281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788d44d748-44r7k" event={"ID":"3e84a46f-0d53-44ae-9db2-decbcc1094f0","Type":"ContainerDied","Data":"48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590"}
Sep 18 18:02:20.571350 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.571332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788d44d748-44r7k" event={"ID":"3e84a46f-0d53-44ae-9db2-decbcc1094f0","Type":"ContainerDied","Data":"ddcf33bf296e1e904262ad67280888a99361bbb7cb14ce3800beb7cd0044b65e"}
Sep 18 18:02:20.571462 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.571355    2582 scope.go:117] "RemoveContainer" containerID="48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590"
Sep 18 18:02:20.579588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.579572    2582 scope.go:117] "RemoveContainer" containerID="48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590"
Sep 18 18:02:20.579860 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:02:20.579843    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590\": container with ID starting with 48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590 not found: ID does not exist" containerID="48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590"
Sep 18 18:02:20.579909 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.579868    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590"} err="failed to get container status \"48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590\": rpc error: code = NotFound desc = could not find container \"48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590\": container with ID starting with 48d395ab36bb9fd7b623d3e3c3c26602d6654d5130c5eaa18edb12c95d05e590 not found: ID does not exist"
Sep 18 18:02:20.599639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.599604    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-788d44d748-44r7k"]
Sep 18 18:02:20.605537 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.605509    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-788d44d748-44r7k"]
Sep 18 18:02:20.919796 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:20.919718    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e84a46f-0d53-44ae-9db2-decbcc1094f0" path="/var/lib/kubelet/pods/3e84a46f-0d53-44ae-9db2-decbcc1094f0/volumes"
Sep 18 18:02:34.632786 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:34.632750    2582 generic.go:356] "Generic (PLEG): container finished" podID="f85d3569-b8be-47de-894a-7f2141464023" containerID="f00f9365c68782bfdb130daf29c04ca837fee830b6d3aa66061cba263d421d87" exitCode=0
Sep 18 18:02:34.633169 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:34.632817    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4" event={"ID":"f85d3569-b8be-47de-894a-7f2141464023","Type":"ContainerDied","Data":"f00f9365c68782bfdb130daf29c04ca837fee830b6d3aa66061cba263d421d87"}
Sep 18 18:02:34.633169 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:34.633116    2582 scope.go:117] "RemoveContainer" containerID="f00f9365c68782bfdb130daf29c04ca837fee830b6d3aa66061cba263d421d87"
Sep 18 18:02:35.639092 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:02:35.639037    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-x2ff4" event={"ID":"f85d3569-b8be-47de-894a-7f2141464023","Type":"ContainerStarted","Data":"f1d025429f682783645275aabbede7413c7d7a8d5eed97c6db609a3dd7f7115c"}
Sep 18 18:03:41.077975 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.077943    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"]
Sep 18 18:03:41.078597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.078409    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a3834a-cc44-470d-8af5-665d4a6e5463" containerName="registry"
Sep 18 18:03:41.078597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.078429    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a3834a-cc44-470d-8af5-665d4a6e5463" containerName="registry"
Sep 18 18:03:41.078597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.078455    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e84a46f-0d53-44ae-9db2-decbcc1094f0" containerName="console"
Sep 18 18:03:41.078597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.078464    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e84a46f-0d53-44ae-9db2-decbcc1094f0" containerName="console"
Sep 18 18:03:41.078597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.078541    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a3834a-cc44-470d-8af5-665d4a6e5463" containerName="registry"
Sep 18 18:03:41.078597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.078553    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e84a46f-0d53-44ae-9db2-decbcc1094f0" containerName="console"
Sep 18 18:03:41.080757 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.080735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.083166 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.083145    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6c97j\""
Sep 18 18:03:41.083274 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.083252    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:03:41.083966 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.083946    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:03:41.090591 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.090571    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"]
Sep 18 18:03:41.247020 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.246979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4hc4\" (UniqueName: \"kubernetes.io/projected/0355149a-add3-4e2b-9b77-c2300ae51ab4-kube-api-access-j4hc4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.247020 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.247025    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.247265 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.247152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.348526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.348444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4hc4\" (UniqueName: \"kubernetes.io/projected/0355149a-add3-4e2b-9b77-c2300ae51ab4-kube-api-access-j4hc4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.348526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.348488    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.348684 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.348550    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.348960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.348937    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.349000 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.348958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.357092 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.357071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4hc4\" (UniqueName: \"kubernetes.io/projected/0355149a-add3-4e2b-9b77-c2300ae51ab4-kube-api-access-j4hc4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.390498 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.390477    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:03:41.665585 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.665504    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"]
Sep 18 18:03:41.669196 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:03:41.669168    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0355149a_add3_4e2b_9b77_c2300ae51ab4.slice/crio-edefcf83143c2760d073cc04625ccb2900d84d5c8eb4b23a42969076916d85f7 WatchSource:0}: Error finding container edefcf83143c2760d073cc04625ccb2900d84d5c8eb4b23a42969076916d85f7: Status 404 returned error can't find the container with id edefcf83143c2760d073cc04625ccb2900d84d5c8eb4b23a42969076916d85f7
Sep 18 18:03:41.897096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:41.897039    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k" event={"ID":"0355149a-add3-4e2b-9b77-c2300ae51ab4","Type":"ContainerStarted","Data":"edefcf83143c2760d073cc04625ccb2900d84d5c8eb4b23a42969076916d85f7"}
Sep 18 18:03:48.927420 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:48.927345    2582 generic.go:356] "Generic (PLEG): container finished" podID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerID="70b4b6ae0f90e4b1d8fa879e3e51b07b6781b3ce8055da36cfd31b88848513a6" exitCode=0
Sep 18 18:03:48.927747 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:48.927433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k" event={"ID":"0355149a-add3-4e2b-9b77-c2300ae51ab4","Type":"ContainerDied","Data":"70b4b6ae0f90e4b1d8fa879e3e51b07b6781b3ce8055da36cfd31b88848513a6"}
Sep 18 18:03:52.944300 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:52.944269    2582 generic.go:356] "Generic (PLEG): container finished" podID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerID="dab47b7687154474986f81d485b6bae4b01691682bfbfa15df119a6c92a04a45" exitCode=0
Sep 18 18:03:52.944666 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:03:52.944308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k" event={"ID":"0355149a-add3-4e2b-9b77-c2300ae51ab4","Type":"ContainerDied","Data":"dab47b7687154474986f81d485b6bae4b01691682bfbfa15df119a6c92a04a45"}
Sep 18 18:04:01.983252 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:01.983212    2582 generic.go:356] "Generic (PLEG): container finished" podID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerID="8b362d6788691f8bedd02e60b66111345a9ac780938cae8440a4a8baae000e02" exitCode=0
Sep 18 18:04:01.983598 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:01.983294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k" event={"ID":"0355149a-add3-4e2b-9b77-c2300ae51ab4","Type":"ContainerDied","Data":"8b362d6788691f8bedd02e60b66111345a9ac780938cae8440a4a8baae000e02"}
Sep 18 18:04:03.129300 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.129265    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:04:03.230272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.230247    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-bundle\") pod \"0355149a-add3-4e2b-9b77-c2300ae51ab4\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") "
Sep 18 18:04:03.230416 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.230282    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-util\") pod \"0355149a-add3-4e2b-9b77-c2300ae51ab4\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") "
Sep 18 18:04:03.230416 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.230327    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4hc4\" (UniqueName: \"kubernetes.io/projected/0355149a-add3-4e2b-9b77-c2300ae51ab4-kube-api-access-j4hc4\") pod \"0355149a-add3-4e2b-9b77-c2300ae51ab4\" (UID: \"0355149a-add3-4e2b-9b77-c2300ae51ab4\") "
Sep 18 18:04:03.230853 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.230827    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-bundle" (OuterVolumeSpecName: "bundle") pod "0355149a-add3-4e2b-9b77-c2300ae51ab4" (UID: "0355149a-add3-4e2b-9b77-c2300ae51ab4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:03.232840 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.232816    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0355149a-add3-4e2b-9b77-c2300ae51ab4-kube-api-access-j4hc4" (OuterVolumeSpecName: "kube-api-access-j4hc4") pod "0355149a-add3-4e2b-9b77-c2300ae51ab4" (UID: "0355149a-add3-4e2b-9b77-c2300ae51ab4"). InnerVolumeSpecName "kube-api-access-j4hc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:03.234450 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.234429    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-util" (OuterVolumeSpecName: "util") pod "0355149a-add3-4e2b-9b77-c2300ae51ab4" (UID: "0355149a-add3-4e2b-9b77-c2300ae51ab4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:03.330946 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.330919    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4hc4\" (UniqueName: \"kubernetes.io/projected/0355149a-add3-4e2b-9b77-c2300ae51ab4-kube-api-access-j4hc4\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.330946 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.330945    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.331129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.330954    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0355149a-add3-4e2b-9b77-c2300ae51ab4-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.994484 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.994449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k" event={"ID":"0355149a-add3-4e2b-9b77-c2300ae51ab4","Type":"ContainerDied","Data":"edefcf83143c2760d073cc04625ccb2900d84d5c8eb4b23a42969076916d85f7"}
Sep 18 18:04:03.994655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.994493    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edefcf83143c2760d073cc04625ccb2900d84d5c8eb4b23a42969076916d85f7"
Sep 18 18:04:03.994655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:03.994492    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540l5m6k"
Sep 18 18:04:16.096463 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096427    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"]
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096712    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="pull"
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096724    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="pull"
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096753    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="util"
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096758    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="util"
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096779    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="extract"
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096785    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="extract"
Sep 18 18:04:16.096899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.096829    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0355149a-add3-4e2b-9b77-c2300ae51ab4" containerName="extract"
Sep 18 18:04:16.098929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.098909    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.101753 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.101730    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6c97j\""
Sep 18 18:04:16.102190 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.102175    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:04:16.102750 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.102737    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:04:16.116130 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.116105    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"]
Sep 18 18:04:16.223101 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.223065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.223258 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.223116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxhg8\" (UniqueName: \"kubernetes.io/projected/a249706b-62b1-482d-8096-d1cd3da82b70-kube-api-access-bxhg8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.223258 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.223153    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.324251 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.324217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.324433 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.324270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxhg8\" (UniqueName: \"kubernetes.io/projected/a249706b-62b1-482d-8096-d1cd3da82b70-kube-api-access-bxhg8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.324433 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.324344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.324684 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.324664    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.324723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.324683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.333559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.333525    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxhg8\" (UniqueName: \"kubernetes.io/projected/a249706b-62b1-482d-8096-d1cd3da82b70-kube-api-access-bxhg8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.408238 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.408167    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:16.678360 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:16.678337    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"]
Sep 18 18:04:16.680910 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:04:16.680880    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda249706b_62b1_482d_8096_d1cd3da82b70.slice/crio-3eabfe3ca1a3dab78da30bf7b7af633f90995fa24a1c35d0b54e527291ea613b WatchSource:0}: Error finding container 3eabfe3ca1a3dab78da30bf7b7af633f90995fa24a1c35d0b54e527291ea613b: Status 404 returned error can't find the container with id 3eabfe3ca1a3dab78da30bf7b7af633f90995fa24a1c35d0b54e527291ea613b
Sep 18 18:04:17.045342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:17.045309    2582 generic.go:356] "Generic (PLEG): container finished" podID="a249706b-62b1-482d-8096-d1cd3da82b70" containerID="2f7ad184f55ce070131003c67b76da67878b3a8cfab82368cf8dd59ca1b3152d" exitCode=0
Sep 18 18:04:17.045539 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:17.045396    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr" event={"ID":"a249706b-62b1-482d-8096-d1cd3da82b70","Type":"ContainerDied","Data":"2f7ad184f55ce070131003c67b76da67878b3a8cfab82368cf8dd59ca1b3152d"}
Sep 18 18:04:17.045539 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:17.045458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr" event={"ID":"a249706b-62b1-482d-8096-d1cd3da82b70","Type":"ContainerStarted","Data":"3eabfe3ca1a3dab78da30bf7b7af633f90995fa24a1c35d0b54e527291ea613b"}
Sep 18 18:04:20.060119 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:20.060082    2582 generic.go:356] "Generic (PLEG): container finished" podID="a249706b-62b1-482d-8096-d1cd3da82b70" containerID="38f5f094eecec1250428d1a5ffec996a94eb90dcdf410b2d873998d69b2e7cb3" exitCode=0
Sep 18 18:04:20.060509 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:20.060167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr" event={"ID":"a249706b-62b1-482d-8096-d1cd3da82b70","Type":"ContainerDied","Data":"38f5f094eecec1250428d1a5ffec996a94eb90dcdf410b2d873998d69b2e7cb3"}
Sep 18 18:04:21.066466 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:21.066436    2582 generic.go:356] "Generic (PLEG): container finished" podID="a249706b-62b1-482d-8096-d1cd3da82b70" containerID="be5705a572e7e842086390347f1b8e63dc61e1e8dbfcc960fa109f1f75a426bf" exitCode=0
Sep 18 18:04:21.066466 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:21.066477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr" event={"ID":"a249706b-62b1-482d-8096-d1cd3da82b70","Type":"ContainerDied","Data":"be5705a572e7e842086390347f1b8e63dc61e1e8dbfcc960fa109f1f75a426bf"}
Sep 18 18:04:22.197560 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.197539    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:22.276089 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.276042    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-util\") pod \"a249706b-62b1-482d-8096-d1cd3da82b70\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") "
Sep 18 18:04:22.276201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.276108    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxhg8\" (UniqueName: \"kubernetes.io/projected/a249706b-62b1-482d-8096-d1cd3da82b70-kube-api-access-bxhg8\") pod \"a249706b-62b1-482d-8096-d1cd3da82b70\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") "
Sep 18 18:04:22.276201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.276146    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-bundle\") pod \"a249706b-62b1-482d-8096-d1cd3da82b70\" (UID: \"a249706b-62b1-482d-8096-d1cd3da82b70\") "
Sep 18 18:04:22.276533 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.276507    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-bundle" (OuterVolumeSpecName: "bundle") pod "a249706b-62b1-482d-8096-d1cd3da82b70" (UID: "a249706b-62b1-482d-8096-d1cd3da82b70"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:22.278344 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.278318    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a249706b-62b1-482d-8096-d1cd3da82b70-kube-api-access-bxhg8" (OuterVolumeSpecName: "kube-api-access-bxhg8") pod "a249706b-62b1-482d-8096-d1cd3da82b70" (UID: "a249706b-62b1-482d-8096-d1cd3da82b70"). InnerVolumeSpecName "kube-api-access-bxhg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:22.280805 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.280784    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-util" (OuterVolumeSpecName: "util") pod "a249706b-62b1-482d-8096-d1cd3da82b70" (UID: "a249706b-62b1-482d-8096-d1cd3da82b70"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:22.377424 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.377363    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:22.377424 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.377395    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxhg8\" (UniqueName: \"kubernetes.io/projected/a249706b-62b1-482d-8096-d1cd3da82b70-kube-api-access-bxhg8\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:22.377424 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.377405    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a249706b-62b1-482d-8096-d1cd3da82b70-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:22.578945 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.578907    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5vrsg"]
Sep 18 18:04:22.579261 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579248    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="extract"
Sep 18 18:04:22.579325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579263    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="extract"
Sep 18 18:04:22.579325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579278    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="util"
Sep 18 18:04:22.579325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579283    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="util"
Sep 18 18:04:22.579325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579290    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="pull"
Sep 18 18:04:22.579325 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579296    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="pull"
Sep 18 18:04:22.579489 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.579343    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a249706b-62b1-482d-8096-d1cd3da82b70" containerName="extract"
Sep 18 18:04:22.581097 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.581070    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.585400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.585362    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 18:04:22.586193 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.586180    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-w62z6\""
Sep 18 18:04:22.586312 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.586241    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 18:04:22.602694 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.602671    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5vrsg"]
Sep 18 18:04:22.679730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.679653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd655a2-6174-4ebc-a80e-50089b387644-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5vrsg\" (UID: \"edd655a2-6174-4ebc-a80e-50089b387644\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.679730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.679698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9qjx\" (UniqueName: \"kubernetes.io/projected/edd655a2-6174-4ebc-a80e-50089b387644-kube-api-access-l9qjx\") pod \"cert-manager-webhook-6486ffd997-5vrsg\" (UID: \"edd655a2-6174-4ebc-a80e-50089b387644\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.780521 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.780489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd655a2-6174-4ebc-a80e-50089b387644-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5vrsg\" (UID: \"edd655a2-6174-4ebc-a80e-50089b387644\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.780641 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.780525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9qjx\" (UniqueName: \"kubernetes.io/projected/edd655a2-6174-4ebc-a80e-50089b387644-kube-api-access-l9qjx\") pod \"cert-manager-webhook-6486ffd997-5vrsg\" (UID: \"edd655a2-6174-4ebc-a80e-50089b387644\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.790144 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.790117    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd655a2-6174-4ebc-a80e-50089b387644-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5vrsg\" (UID: \"edd655a2-6174-4ebc-a80e-50089b387644\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.790267 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.790250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9qjx\" (UniqueName: \"kubernetes.io/projected/edd655a2-6174-4ebc-a80e-50089b387644-kube-api-access-l9qjx\") pod \"cert-manager-webhook-6486ffd997-5vrsg\" (UID: \"edd655a2-6174-4ebc-a80e-50089b387644\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:22.911952 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:22.911922    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:23.076135 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:23.076110    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr"
Sep 18 18:04:23.076135 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:23.076084    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnpmvr" event={"ID":"a249706b-62b1-482d-8096-d1cd3da82b70","Type":"ContainerDied","Data":"3eabfe3ca1a3dab78da30bf7b7af633f90995fa24a1c35d0b54e527291ea613b"}
Sep 18 18:04:23.076386 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:23.076169    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3eabfe3ca1a3dab78da30bf7b7af633f90995fa24a1c35d0b54e527291ea613b"
Sep 18 18:04:23.192839 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:23.192497    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5vrsg"]
Sep 18 18:04:23.195248 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:04:23.195223    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedd655a2_6174_4ebc_a80e_50089b387644.slice/crio-c99c9899b604fe3c4480b409dbe8caae8ad7e46b82766d428cb9de3768e79e29 WatchSource:0}: Error finding container c99c9899b604fe3c4480b409dbe8caae8ad7e46b82766d428cb9de3768e79e29: Status 404 returned error can't find the container with id c99c9899b604fe3c4480b409dbe8caae8ad7e46b82766d428cb9de3768e79e29
Sep 18 18:04:24.081678 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:24.081645    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg" event={"ID":"edd655a2-6174-4ebc-a80e-50089b387644","Type":"ContainerStarted","Data":"c99c9899b604fe3c4480b409dbe8caae8ad7e46b82766d428cb9de3768e79e29"}
Sep 18 18:04:25.264667 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.264632    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"]
Sep 18 18:04:25.268773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.268747    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.273104 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.273077    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4bmnc\""
Sep 18 18:04:25.274069 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.274030    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 18:04:25.274205 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.274084    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:04:25.284216 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.284188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"]
Sep 18 18:04:25.401765 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.401725    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7xhm\" (UniqueName: \"kubernetes.io/projected/7da96638-20d0-4ad0-8e28-2d73a223a718-kube-api-access-g7xhm\") pod \"openshift-lws-operator-fd8ccff4c-jqfzz\" (UID: \"7da96638-20d0-4ad0-8e28-2d73a223a718\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.401955 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.401846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7da96638-20d0-4ad0-8e28-2d73a223a718-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jqfzz\" (UID: \"7da96638-20d0-4ad0-8e28-2d73a223a718\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.503356 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.503317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7xhm\" (UniqueName: \"kubernetes.io/projected/7da96638-20d0-4ad0-8e28-2d73a223a718-kube-api-access-g7xhm\") pod \"openshift-lws-operator-fd8ccff4c-jqfzz\" (UID: \"7da96638-20d0-4ad0-8e28-2d73a223a718\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.503544 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.503431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7da96638-20d0-4ad0-8e28-2d73a223a718-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jqfzz\" (UID: \"7da96638-20d0-4ad0-8e28-2d73a223a718\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.503915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.503845    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7da96638-20d0-4ad0-8e28-2d73a223a718-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jqfzz\" (UID: \"7da96638-20d0-4ad0-8e28-2d73a223a718\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.532575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.532549    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7xhm\" (UniqueName: \"kubernetes.io/projected/7da96638-20d0-4ad0-8e28-2d73a223a718-kube-api-access-g7xhm\") pod \"openshift-lws-operator-fd8ccff4c-jqfzz\" (UID: \"7da96638-20d0-4ad0-8e28-2d73a223a718\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:25.580452 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:25.580417    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"
Sep 18 18:04:26.589852 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:26.589823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz"]
Sep 18 18:04:26.591622 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:04:26.591592    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7da96638_20d0_4ad0_8e28_2d73a223a718.slice/crio-ec8abd7762406e39acdc5e44694857a3d9315bdd30291fd5d03e0b42e4053dc4 WatchSource:0}: Error finding container ec8abd7762406e39acdc5e44694857a3d9315bdd30291fd5d03e0b42e4053dc4: Status 404 returned error can't find the container with id ec8abd7762406e39acdc5e44694857a3d9315bdd30291fd5d03e0b42e4053dc4
Sep 18 18:04:27.096784 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:27.096753    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg" event={"ID":"edd655a2-6174-4ebc-a80e-50089b387644","Type":"ContainerStarted","Data":"4a4e83fc9e40520cf3b302e2902a180e896d019a15d86c6290b23210dad2c167"}
Sep 18 18:04:27.096973 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:27.096832    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:27.098229 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:27.098205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz" event={"ID":"7da96638-20d0-4ad0-8e28-2d73a223a718","Type":"ContainerStarted","Data":"ec8abd7762406e39acdc5e44694857a3d9315bdd30291fd5d03e0b42e4053dc4"}
Sep 18 18:04:27.115172 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:27.115132    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg" podStartSLOduration=1.933690844 podStartE2EDuration="5.115119684s" totalImagesPullingTime="3.18142884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:27.1132287 +0000 UTC m=+250.540441492" watchObservedRunningTime="2026-09-18 18:04:27.115119684 +0000 UTC m=+250.542332437"
Sep 18 18:04:29.107916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:29.107822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz" event={"ID":"7da96638-20d0-4ad0-8e28-2d73a223a718","Type":"ContainerStarted","Data":"f23c7f1656058aa50bf798021ee5e8ab63139f17120e10e23dcb09159c08c421"}
Sep 18 18:04:29.127146 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:29.127098    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqfzz" podStartSLOduration=1.9022261839999999 podStartE2EDuration="4.127081921s" totalImagesPullingTime="2.224855737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:29.126494497 +0000 UTC m=+252.553707264" watchObservedRunningTime="2026-09-18 18:04:29.127081921 +0000 UTC m=+252.554294674"
Sep 18 18:04:31.450851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.450818    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"]
Sep 18 18:04:31.497883 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.497839    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"]
Sep 18 18:04:31.498076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.497908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.500914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.500895    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:04:31.501696 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.501668    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:04:31.501696 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.501689    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6c97j\""
Sep 18 18:04:31.657678 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.657643    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.657843 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.657734    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgs4z\" (UniqueName: \"kubernetes.io/projected/4418c3b2-adad-436f-8364-1347913f4a20-kube-api-access-zgs4z\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.657843 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.657758    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.759184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.759097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgs4z\" (UniqueName: \"kubernetes.io/projected/4418c3b2-adad-436f-8364-1347913f4a20-kube-api-access-zgs4z\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.759184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.759136    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.759184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.759179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.759523 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.759505    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.759558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.759537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.772815 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.772788    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgs4z\" (UniqueName: \"kubernetes.io/projected/4418c3b2-adad-436f-8364-1347913f4a20-kube-api-access-zgs4z\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:31.806628 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:31.806596    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:32.086604 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:32.086580    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"]
Sep 18 18:04:32.088565 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:04:32.088537    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4418c3b2_adad_436f_8364_1347913f4a20.slice/crio-da8f6984b04820d620b2985aad077208e3853cc30e49257196b115829cab05f0 WatchSource:0}: Error finding container da8f6984b04820d620b2985aad077208e3853cc30e49257196b115829cab05f0: Status 404 returned error can't find the container with id da8f6984b04820d620b2985aad077208e3853cc30e49257196b115829cab05f0
Sep 18 18:04:32.123810 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:32.123785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x" event={"ID":"4418c3b2-adad-436f-8364-1347913f4a20","Type":"ContainerStarted","Data":"da8f6984b04820d620b2985aad077208e3853cc30e49257196b115829cab05f0"}
Sep 18 18:04:33.105159 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:33.105126    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5vrsg"
Sep 18 18:04:33.130810 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:33.130774    2582 generic.go:356] "Generic (PLEG): container finished" podID="4418c3b2-adad-436f-8364-1347913f4a20" containerID="9db379b82387b1634c5d452d2f3582fbd34e92e3ff308c9a2db8dd0a011c5e2f" exitCode=0
Sep 18 18:04:33.130991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:33.130813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x" event={"ID":"4418c3b2-adad-436f-8364-1347913f4a20","Type":"ContainerDied","Data":"9db379b82387b1634c5d452d2f3582fbd34e92e3ff308c9a2db8dd0a011c5e2f"}
Sep 18 18:04:34.136863 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:34.136828    2582 generic.go:356] "Generic (PLEG): container finished" podID="4418c3b2-adad-436f-8364-1347913f4a20" containerID="6f5374b79f41dfe2391a262e52900f413fdd85e84bf8ca181e43bbdd9feafd13" exitCode=0
Sep 18 18:04:34.137235 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:34.136898    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x" event={"ID":"4418c3b2-adad-436f-8364-1347913f4a20","Type":"ContainerDied","Data":"6f5374b79f41dfe2391a262e52900f413fdd85e84bf8ca181e43bbdd9feafd13"}
Sep 18 18:04:35.146338 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:35.146296    2582 generic.go:356] "Generic (PLEG): container finished" podID="4418c3b2-adad-436f-8364-1347913f4a20" containerID="7dc0cd12ccb26ba90cc1bd93ce2c9c3c2ed54aa95e210ec382146119d3525ee3" exitCode=0
Sep 18 18:04:35.146986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:35.146959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x" event={"ID":"4418c3b2-adad-436f-8364-1347913f4a20","Type":"ContainerDied","Data":"7dc0cd12ccb26ba90cc1bd93ce2c9c3c2ed54aa95e210ec382146119d3525ee3"}
Sep 18 18:04:36.281530 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.281508    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:36.402192 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.402118    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgs4z\" (UniqueName: \"kubernetes.io/projected/4418c3b2-adad-436f-8364-1347913f4a20-kube-api-access-zgs4z\") pod \"4418c3b2-adad-436f-8364-1347913f4a20\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") "
Sep 18 18:04:36.402192 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.402156    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-util\") pod \"4418c3b2-adad-436f-8364-1347913f4a20\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") "
Sep 18 18:04:36.402192 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.402185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-bundle\") pod \"4418c3b2-adad-436f-8364-1347913f4a20\" (UID: \"4418c3b2-adad-436f-8364-1347913f4a20\") "
Sep 18 18:04:36.403040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.403005    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-bundle" (OuterVolumeSpecName: "bundle") pod "4418c3b2-adad-436f-8364-1347913f4a20" (UID: "4418c3b2-adad-436f-8364-1347913f4a20"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:36.404318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.404285    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4418c3b2-adad-436f-8364-1347913f4a20-kube-api-access-zgs4z" (OuterVolumeSpecName: "kube-api-access-zgs4z") pod "4418c3b2-adad-436f-8364-1347913f4a20" (UID: "4418c3b2-adad-436f-8364-1347913f4a20"). InnerVolumeSpecName "kube-api-access-zgs4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:36.410667 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.410643    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-util" (OuterVolumeSpecName: "util") pod "4418c3b2-adad-436f-8364-1347913f4a20" (UID: "4418c3b2-adad-436f-8364-1347913f4a20"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:36.502693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.502664    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgs4z\" (UniqueName: \"kubernetes.io/projected/4418c3b2-adad-436f-8364-1347913f4a20-kube-api-access-zgs4z\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:36.502693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.502688    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:36.502693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:36.502698    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4418c3b2-adad-436f-8364-1347913f4a20-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:37.158346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:37.158313    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x" event={"ID":"4418c3b2-adad-436f-8364-1347913f4a20","Type":"ContainerDied","Data":"da8f6984b04820d620b2985aad077208e3853cc30e49257196b115829cab05f0"}
Sep 18 18:04:37.158346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:37.158349    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aff2p2x"
Sep 18 18:04:37.158543 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:37.158358    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da8f6984b04820d620b2985aad077208e3853cc30e49257196b115829cab05f0"
Sep 18 18:04:48.620323 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.620285    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"]
Sep 18 18:04:48.621604 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.621574    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="util"
Sep 18 18:04:48.621766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.621755    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="util"
Sep 18 18:04:48.621859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.621849    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="pull"
Sep 18 18:04:48.621937 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.621929    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="pull"
Sep 18 18:04:48.622083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.622071    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="extract"
Sep 18 18:04:48.622171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.622162    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="extract"
Sep 18 18:04:48.622398 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.622385    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4418c3b2-adad-436f-8364-1347913f4a20" containerName="extract"
Sep 18 18:04:48.625588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.625569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.631448 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.631426    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:04:48.631593 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.631527    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6c97j\""
Sep 18 18:04:48.631721 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.631588    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:04:48.633722 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.633157    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"]
Sep 18 18:04:48.806375 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.806331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.806550 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.806396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.806550 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.806476    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf28k\" (UniqueName: \"kubernetes.io/projected/d58239aa-1959-4eb7-93fb-8074dd200a4a-kube-api-access-rf28k\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.906970 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.906874    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.906970 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.906932    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf28k\" (UniqueName: \"kubernetes.io/projected/d58239aa-1959-4eb7-93fb-8074dd200a4a-kube-api-access-rf28k\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.907198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.906995    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.907486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.907430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.907486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.907432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.921497 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.921467    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf28k\" (UniqueName: \"kubernetes.io/projected/d58239aa-1959-4eb7-93fb-8074dd200a4a-kube-api-access-rf28k\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:48.937100 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:48.937069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:49.129031 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.128994    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"]
Sep 18 18:04:49.131444 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.131428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.135987 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.135967    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 18:04:49.136318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.136301    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 18:04:49.136454 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.136436    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 18:04:49.136812 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.136795    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cj7q6\""
Sep 18 18:04:49.137274 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.137244    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 18:04:49.167028 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.166977    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"]
Sep 18 18:04:49.241369 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.241342    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"]
Sep 18 18:04:49.244062 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:04:49.244022    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58239aa_1959_4eb7_93fb_8074dd200a4a.slice/crio-35abe292b747d9fa0a8267a443bfe31426f696dff6dfa671cfea20186b405911 WatchSource:0}: Error finding container 35abe292b747d9fa0a8267a443bfe31426f696dff6dfa671cfea20186b405911: Status 404 returned error can't find the container with id 35abe292b747d9fa0a8267a443bfe31426f696dff6dfa671cfea20186b405911
Sep 18 18:04:49.310416 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.310380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.310523 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.310460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-manifests\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.310574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.310522    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-tmp\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.310574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.310546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.310574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.310561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zj5\" (UniqueName: \"kubernetes.io/projected/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-kube-api-access-58zj5\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.411967 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.411930    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.412149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.411985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-manifests\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.412149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.412030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-tmp\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.412149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.412085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.412149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.412110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zj5\" (UniqueName: \"kubernetes.io/projected/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-kube-api-access-58zj5\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.412475 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.412449    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-manifests\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.414821 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.414786    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.414933 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.414911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.414933 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.414917    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-tmp\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.427341 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.427293    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zj5\" (UniqueName: \"kubernetes.io/projected/5a41a42a-ec09-4a7b-8566-4cc36e482b9a-kube-api-access-58zj5\") pod \"opendatahub-operator-controller-manager-57cbb766c6-dlxnk\" (UID: \"5a41a42a-ec09-4a7b-8566-4cc36e482b9a\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.444292 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.443622    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:49.735663 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:49.735640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"]
Sep 18 18:04:49.736822 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:04:49.736791    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a41a42a_ec09_4a7b_8566_4cc36e482b9a.slice/crio-73e6bb3c816e41f9d047349704bbf1e6124926f549d45f664a67a0be7965d4c6 WatchSource:0}: Error finding container 73e6bb3c816e41f9d047349704bbf1e6124926f549d45f664a67a0be7965d4c6: Status 404 returned error can't find the container with id 73e6bb3c816e41f9d047349704bbf1e6124926f549d45f664a67a0be7965d4c6
Sep 18 18:04:50.220183 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:50.220148    2582 generic.go:356] "Generic (PLEG): container finished" podID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerID="0e64b8dfa43138d636f2ec2cbf176bcf297aef0ca00390b68c8f152100c83401" exitCode=0
Sep 18 18:04:50.220346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:50.220242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj" event={"ID":"d58239aa-1959-4eb7-93fb-8074dd200a4a","Type":"ContainerDied","Data":"0e64b8dfa43138d636f2ec2cbf176bcf297aef0ca00390b68c8f152100c83401"}
Sep 18 18:04:50.220346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:50.220293    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj" event={"ID":"d58239aa-1959-4eb7-93fb-8074dd200a4a","Type":"ContainerStarted","Data":"35abe292b747d9fa0a8267a443bfe31426f696dff6dfa671cfea20186b405911"}
Sep 18 18:04:50.221658 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:50.221638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk" event={"ID":"5a41a42a-ec09-4a7b-8566-4cc36e482b9a","Type":"ContainerStarted","Data":"73e6bb3c816e41f9d047349704bbf1e6124926f549d45f664a67a0be7965d4c6"}
Sep 18 18:04:51.231160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:51.231078    2582 generic.go:356] "Generic (PLEG): container finished" podID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerID="3babd5470e17ca420259b4f09f0471f7ce057dcfa1759a12cd6af1010f9f143b" exitCode=0
Sep 18 18:04:51.231160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:51.231125    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj" event={"ID":"d58239aa-1959-4eb7-93fb-8074dd200a4a","Type":"ContainerDied","Data":"3babd5470e17ca420259b4f09f0471f7ce057dcfa1759a12cd6af1010f9f143b"}
Sep 18 18:04:52.240021 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:52.239990    2582 generic.go:356] "Generic (PLEG): container finished" podID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerID="a84c6046e0974307b2e2103527c125b92501b3ffac43759897ff21140ac28848" exitCode=0
Sep 18 18:04:52.240450 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:52.240090    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj" event={"ID":"d58239aa-1959-4eb7-93fb-8074dd200a4a","Type":"ContainerDied","Data":"a84c6046e0974307b2e2103527c125b92501b3ffac43759897ff21140ac28848"}
Sep 18 18:04:53.246881 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.246788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk" event={"ID":"5a41a42a-ec09-4a7b-8566-4cc36e482b9a","Type":"ContainerStarted","Data":"10763842e0ea52ec4a5fa48bc429e90481fc6026a97a329040ba1e1d787b04ba"}
Sep 18 18:04:53.376203 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.376179    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:53.446852 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.446818    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf28k\" (UniqueName: \"kubernetes.io/projected/d58239aa-1959-4eb7-93fb-8074dd200a4a-kube-api-access-rf28k\") pod \"d58239aa-1959-4eb7-93fb-8074dd200a4a\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") "
Sep 18 18:04:53.447008 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.446866    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-bundle\") pod \"d58239aa-1959-4eb7-93fb-8074dd200a4a\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") "
Sep 18 18:04:53.447008 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.446907    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-util\") pod \"d58239aa-1959-4eb7-93fb-8074dd200a4a\" (UID: \"d58239aa-1959-4eb7-93fb-8074dd200a4a\") "
Sep 18 18:04:53.447650 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.447579    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-bundle" (OuterVolumeSpecName: "bundle") pod "d58239aa-1959-4eb7-93fb-8074dd200a4a" (UID: "d58239aa-1959-4eb7-93fb-8074dd200a4a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:53.449087 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.449063    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58239aa-1959-4eb7-93fb-8074dd200a4a-kube-api-access-rf28k" (OuterVolumeSpecName: "kube-api-access-rf28k") pod "d58239aa-1959-4eb7-93fb-8074dd200a4a" (UID: "d58239aa-1959-4eb7-93fb-8074dd200a4a"). InnerVolumeSpecName "kube-api-access-rf28k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:53.452687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.452647    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-util" (OuterVolumeSpecName: "util") pod "d58239aa-1959-4eb7-93fb-8074dd200a4a" (UID: "d58239aa-1959-4eb7-93fb-8074dd200a4a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:04:53.547991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.547964    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf28k\" (UniqueName: \"kubernetes.io/projected/d58239aa-1959-4eb7-93fb-8074dd200a4a-kube-api-access-rf28k\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:53.547991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.547991    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:53.548172 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:53.548000    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d58239aa-1959-4eb7-93fb-8074dd200a4a-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:04:54.253898 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:54.253871    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj"
Sep 18 18:04:54.254327 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:54.253868    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b04n9kj" event={"ID":"d58239aa-1959-4eb7-93fb-8074dd200a4a","Type":"ContainerDied","Data":"35abe292b747d9fa0a8267a443bfe31426f696dff6dfa671cfea20186b405911"}
Sep 18 18:04:54.254327 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:54.253993    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35abe292b747d9fa0a8267a443bfe31426f696dff6dfa671cfea20186b405911"
Sep 18 18:04:54.255614 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:54.255591    2582 generic.go:356] "Generic (PLEG): container finished" podID="5a41a42a-ec09-4a7b-8566-4cc36e482b9a" containerID="10763842e0ea52ec4a5fa48bc429e90481fc6026a97a329040ba1e1d787b04ba" exitCode=0
Sep 18 18:04:54.255734 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:54.255621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk" event={"ID":"5a41a42a-ec09-4a7b-8566-4cc36e482b9a","Type":"ContainerDied","Data":"10763842e0ea52ec4a5fa48bc429e90481fc6026a97a329040ba1e1d787b04ba"}
Sep 18 18:04:55.263496 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:55.263463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk" event={"ID":"5a41a42a-ec09-4a7b-8566-4cc36e482b9a","Type":"ContainerStarted","Data":"aaa1f07d36db3348a930cb1b0ddd23100200849928353f6cb3f60d1c511b06de"}
Sep 18 18:04:55.263972 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:55.263561    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:04:55.294010 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:04:55.293941    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk" podStartSLOduration=1.662795233 podStartE2EDuration="6.293925815s" totalImagesPullingTime="3.61787028s" totalInitContainerRuntime="1.013260302s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:55.293388723 +0000 UTC m=+278.720601497" watchObservedRunningTime="2026-09-18 18:04:55.293925815 +0000 UTC m=+278.721138568"
Sep 18 18:05:06.270097 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:06.270039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-dlxnk"
Sep 18 18:05:09.485398 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485354    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"]
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485679    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="extract"
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485692    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="extract"
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485705    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="pull"
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485710    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="pull"
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485722    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="util"
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485727    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="util"
Sep 18 18:05:09.485800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.485782    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58239aa-1959-4eb7-93fb-8074dd200a4a" containerName="extract"
Sep 18 18:05:09.488535 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.488505    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.491570 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.491544    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 18:05:09.491943 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.491913    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 18:05:09.530588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.530555    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"]
Sep 18 18:05:09.576044 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.576005    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr2tg\" (UniqueName: \"kubernetes.io/projected/e0fb050f-e042-4027-bfa4-474b12a84c68-kube-api-access-cr2tg\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.576044 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.576061    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0fb050f-e042-4027-bfa4-474b12a84c68-tmp\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.576273 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.576115    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fb050f-e042-4027-bfa4-474b12a84c68-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.677171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.677134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr2tg\" (UniqueName: \"kubernetes.io/projected/e0fb050f-e042-4027-bfa4-474b12a84c68-kube-api-access-cr2tg\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.677171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.677170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0fb050f-e042-4027-bfa4-474b12a84c68-tmp\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.677393 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.677194    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fb050f-e042-4027-bfa4-474b12a84c68-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.677393 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:05:09.677300    2582 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 18:05:09.677393 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:05:09.677356    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0fb050f-e042-4027-bfa4-474b12a84c68-tls-certs podName:e0fb050f-e042-4027-bfa4-474b12a84c68 nodeName:}" failed. No retries permitted until 2026-09-18 18:05:10.17734002 +0000 UTC m=+293.604552766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e0fb050f-e042-4027-bfa4-474b12a84c68-tls-certs") pod "kube-auth-proxy-9fbf455f6-rs4vh" (UID: "e0fb050f-e042-4027-bfa4-474b12a84c68") : secret "kube-auth-proxy-tls" not found
Sep 18 18:05:09.679588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.679565    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0fb050f-e042-4027-bfa4-474b12a84c68-tmp\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:09.703035 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:09.703007    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr2tg\" (UniqueName: \"kubernetes.io/projected/e0fb050f-e042-4027-bfa4-474b12a84c68-kube-api-access-cr2tg\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:10.182792 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:10.182752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fb050f-e042-4027-bfa4-474b12a84c68-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:10.185474 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:10.185444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fb050f-e042-4027-bfa4-474b12a84c68-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-rs4vh\" (UID: \"e0fb050f-e042-4027-bfa4-474b12a84c68\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:10.399156 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:10.399125    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"
Sep 18 18:05:10.679444 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:10.679372    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh"]
Sep 18 18:05:10.682158 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:05:10.682122    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0fb050f_e042_4027_bfa4_474b12a84c68.slice/crio-746c0c78e3545ac239e2c508299d05d8ae3cfb7a7cc2add1c1e3820ba2fafaa8 WatchSource:0}: Error finding container 746c0c78e3545ac239e2c508299d05d8ae3cfb7a7cc2add1c1e3820ba2fafaa8: Status 404 returned error can't find the container with id 746c0c78e3545ac239e2c508299d05d8ae3cfb7a7cc2add1c1e3820ba2fafaa8
Sep 18 18:05:11.338824 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:11.338784    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh" event={"ID":"e0fb050f-e042-4027-bfa4-474b12a84c68","Type":"ContainerStarted","Data":"746c0c78e3545ac239e2c508299d05d8ae3cfb7a7cc2add1c1e3820ba2fafaa8"}
Sep 18 18:05:14.362958 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:14.362915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh" event={"ID":"e0fb050f-e042-4027-bfa4-474b12a84c68","Type":"ContainerStarted","Data":"5ca61e082554ca1d60220e201d26f334154099846dcd6a5f318202b249557239"}
Sep 18 18:05:18.046399 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.046335    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-rs4vh" podStartSLOduration=5.843155536 podStartE2EDuration="9.046311082s" totalImagesPullingTime="3.203155546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:14.386181312 +0000 UTC m=+297.813394065" watchObservedRunningTime="2026-09-18 18:05:18.046311082 +0000 UTC m=+301.473523836"
Sep 18 18:05:18.048204 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.048183    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"]
Sep 18 18:05:18.051543 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.051526    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.056931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.056908    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:05:18.057038 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.056985    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sfcsh\""
Sep 18 18:05:18.072877 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.072851    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"]
Sep 18 18:05:18.156184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2cef883-e888-4392-ba52-2fe8fcbabe34-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156196    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156320    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cww2k\" (UniqueName: \"kubernetes.io/projected/a2cef883-e888-4392-ba52-2fe8fcbabe34-kube-api-access-cww2k\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156590 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156590 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.156590 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.156501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257702 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257661    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257702 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257728    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cww2k\" (UniqueName: \"kubernetes.io/projected/a2cef883-e888-4392-ba52-2fe8fcbabe34-kube-api-access-cww2k\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2cef883-e888-4392-ba52-2fe8fcbabe34-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.257990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.257977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.258398 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.258329    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.258578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.258547    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.258971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.258943    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.258971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.258581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.259176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.259102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.259580 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.259556    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2cef883-e888-4392-ba52-2fe8fcbabe34-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.261334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.261309    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.261654 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.261633    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.267670 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.267648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cww2k\" (UniqueName: \"kubernetes.io/projected/a2cef883-e888-4392-ba52-2fe8fcbabe34-kube-api-access-cww2k\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.269002 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.268982    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2cef883-e888-4392-ba52-2fe8fcbabe34-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6\" (UID: \"a2cef883-e888-4392-ba52-2fe8fcbabe34\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.362307 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.362236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:18.645039 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.644961    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"]
Sep 18 18:05:18.648326 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:05:18.648294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2cef883_e888_4392_ba52_2fe8fcbabe34.slice/crio-9603ef9f79322b4e738ada39440dfa37eb89196240833913dff8039a52cf0d90 WatchSource:0}: Error finding container 9603ef9f79322b4e738ada39440dfa37eb89196240833913dff8039a52cf0d90: Status 404 returned error can't find the container with id 9603ef9f79322b4e738ada39440dfa37eb89196240833913dff8039a52cf0d90
Sep 18 18:05:18.773394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.773362    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:05:18.777100 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.777078    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:05:18.785089 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:18.785067    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 18:05:19.388606 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.388561    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6" event={"ID":"a2cef883-e888-4392-ba52-2fe8fcbabe34","Type":"ContainerStarted","Data":"9603ef9f79322b4e738ada39440dfa37eb89196240833913dff8039a52cf0d90"}
Sep 18 18:05:19.907269 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.907230    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rtwmn"]
Sep 18 18:05:19.910778 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.910746    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:19.913465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.913438    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 18:05:19.913590 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.913449    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lwdtm\""
Sep 18 18:05:19.922558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.922526    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rtwmn"]
Sep 18 18:05:19.976526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.976480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b9nf\" (UniqueName: \"kubernetes.io/projected/944e07c9-b553-45bc-8438-2989850f4016-kube-api-access-5b9nf\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:19.976710 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:19.976609    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/944e07c9-b553-45bc-8438-2989850f4016-cert\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.078112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.077957    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b9nf\" (UniqueName: \"kubernetes.io/projected/944e07c9-b553-45bc-8438-2989850f4016-kube-api-access-5b9nf\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.078112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.078013    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/944e07c9-b553-45bc-8438-2989850f4016-cert\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.078344 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:05:20.078168    2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 18 18:05:20.078344 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:05:20.078245    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/944e07c9-b553-45bc-8438-2989850f4016-cert podName:944e07c9-b553-45bc-8438-2989850f4016 nodeName:}" failed. No retries permitted until 2026-09-18 18:05:20.578221931 +0000 UTC m=+304.005434667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/944e07c9-b553-45bc-8438-2989850f4016-cert") pod "odh-model-controller-7998c67dc4-rtwmn" (UID: "944e07c9-b553-45bc-8438-2989850f4016") : secret "odh-model-controller-webhook-cert" not found
Sep 18 18:05:20.088211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.088179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b9nf\" (UniqueName: \"kubernetes.io/projected/944e07c9-b553-45bc-8438-2989850f4016-kube-api-access-5b9nf\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.584164 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.584084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/944e07c9-b553-45bc-8438-2989850f4016-cert\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.587471 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.587441    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/944e07c9-b553-45bc-8438-2989850f4016-cert\") pod \"odh-model-controller-7998c67dc4-rtwmn\" (UID: \"944e07c9-b553-45bc-8438-2989850f4016\") " pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.630100 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.630041    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rchwf"]
Sep 18 18:05:20.638916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.638877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.646381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.646337    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 18:05:20.650522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.650497    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rchwf"]
Sep 18 18:05:20.685122 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.685083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4tq\" (UniqueName: \"kubernetes.io/projected/3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806-kube-api-access-fb4tq\") pod \"model-serving-api-54d89bd5fc-rchwf\" (UID: \"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806\") " pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.685306 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.685156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806-tls-certs\") pod \"model-serving-api-54d89bd5fc-rchwf\" (UID: \"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806\") " pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.785974 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.785934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4tq\" (UniqueName: \"kubernetes.io/projected/3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806-kube-api-access-fb4tq\") pod \"model-serving-api-54d89bd5fc-rchwf\" (UID: \"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806\") " pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.786199 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.786008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806-tls-certs\") pod \"model-serving-api-54d89bd5fc-rchwf\" (UID: \"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806\") " pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.788851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.788821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806-tls-certs\") pod \"model-serving-api-54d89bd5fc-rchwf\" (UID: \"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806\") " pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.800281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.800256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4tq\" (UniqueName: \"kubernetes.io/projected/3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806-kube-api-access-fb4tq\") pod \"model-serving-api-54d89bd5fc-rchwf\" (UID: \"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806\") " pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:20.824066 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.824025    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:20.953760 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:20.953716    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:21.636534 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:21.636473    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:05:21.637136 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:21.636552    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:05:21.637136 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:21.636582    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:05:21.843353 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:21.839182    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rchwf"]
Sep 18 18:05:21.843353 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:21.842228    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rtwmn"]
Sep 18 18:05:21.845823 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:21.844564    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:05:21.851124 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:05:21.851093    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944e07c9_b553_45bc_8438_2989850f4016.slice/crio-4bfe6445e788069b419ef4964f0416a45490f9acc497b3d3454aa7b766eb73e4 WatchSource:0}: Error finding container 4bfe6445e788069b419ef4964f0416a45490f9acc497b3d3454aa7b766eb73e4: Status 404 returned error can't find the container with id 4bfe6445e788069b419ef4964f0416a45490f9acc497b3d3454aa7b766eb73e4
Sep 18 18:05:22.409720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:22.409673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn" event={"ID":"944e07c9-b553-45bc-8438-2989850f4016","Type":"ContainerStarted","Data":"4bfe6445e788069b419ef4964f0416a45490f9acc497b3d3454aa7b766eb73e4"}
Sep 18 18:05:22.416794 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:22.415722    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6" event={"ID":"a2cef883-e888-4392-ba52-2fe8fcbabe34","Type":"ContainerStarted","Data":"54e7c65de97b8f23fcfda095f6bce45f97b58660e4489d0ff2246fdfc406557e"}
Sep 18 18:05:22.427243 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:22.427139    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf" event={"ID":"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806","Type":"ContainerStarted","Data":"052c0f45299ea3d21aa09d76399a493d90a4ec8de16faee0b311c637cd8fee3d"}
Sep 18 18:05:22.468813 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:22.468746    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6" podStartSLOduration=1.482936049 podStartE2EDuration="4.468721692s" totalImagesPullingTime="2.985785643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:22.465678106 +0000 UTC m=+305.892890861" watchObservedRunningTime="2026-09-18 18:05:22.468721692 +0000 UTC m=+305.895934447"
Sep 18 18:05:23.363317 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:23.362961    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:23.369756 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:23.369482    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:23.435574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:23.435323    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:23.436698 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:23.436639    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6"
Sep 18 18:05:25.449584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:25.449537    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf" event={"ID":"3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806","Type":"ContainerStarted","Data":"1cc0d19e6c63490788f92f1211cd28ae585e328d631c6c3435f93e2608a5938b"}
Sep 18 18:05:25.450072 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:25.449667    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:25.451583 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:25.451556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn" event={"ID":"944e07c9-b553-45bc-8438-2989850f4016","Type":"ContainerStarted","Data":"88a639a9ce6076b14128ea42882d114a31892a8c8bbceb3054942366dfdb1195"}
Sep 18 18:05:25.467871 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:25.467825    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf" podStartSLOduration=2.39706583 podStartE2EDuration="5.467807225s" totalImagesPullingTime="3.070741395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:25.4657173 +0000 UTC m=+308.892930054" watchObservedRunningTime="2026-09-18 18:05:25.467807225 +0000 UTC m=+308.895019970"
Sep 18 18:05:25.489949 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:25.489889    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn" podStartSLOduration=3.374489771 podStartE2EDuration="6.489869819s" totalImagesPullingTime="3.115380048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:25.487138505 +0000 UTC m=+308.914351260" watchObservedRunningTime="2026-09-18 18:05:25.489869819 +0000 UTC m=+308.917082574"
Sep 18 18:05:26.457122 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:26.457090    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:05:28.320808 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.320773    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"]
Sep 18 18:05:28.327644 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.327627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.333456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.333430    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 18:05:28.333456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.333447    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 18:05:28.333626 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.333458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 18:05:28.333626 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.333454    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-plh74\""
Sep 18 18:05:28.373734 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.373701    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"]
Sep 18 18:05:28.464623 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464578    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93782abd-481e-457f-b67c-5469a42d428e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.464828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vrb6\" (UniqueName: \"kubernetes.io/projected/93782abd-481e-457f-b67c-5469a42d428e-kube-api-access-4vrb6\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.464828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.464828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.464828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93782abd-481e-457f-b67c-5469a42d428e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.464828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.465210 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.464850    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93782abd-481e-457f-b67c-5469a42d428e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566131 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93782abd-481e-457f-b67c-5469a42d428e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566131 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vrb6\" (UniqueName: \"kubernetes.io/projected/93782abd-481e-457f-b67c-5469a42d428e-kube-api-access-4vrb6\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566155    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93782abd-481e-457f-b67c-5469a42d428e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.566394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.566267    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93782abd-481e-457f-b67c-5469a42d428e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.567310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.567280    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93782abd-481e-457f-b67c-5469a42d428e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.570596 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.569417    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.570596 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.569624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.575878 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.575798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93782abd-481e-457f-b67c-5469a42d428e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.575878 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.575855    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93782abd-481e-457f-b67c-5469a42d428e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.597555 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.597524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vrb6\" (UniqueName: \"kubernetes.io/projected/93782abd-481e-457f-b67c-5469a42d428e-kube-api-access-4vrb6\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.597883 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.597860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93782abd-481e-457f-b67c-5469a42d428e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cnvjt\" (UID: \"93782abd-481e-457f-b67c-5469a42d428e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.638628 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.638596    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:28.942501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:28.942432    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"]
Sep 18 18:05:28.952294 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:05:28.952261    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93782abd_481e_457f_b67c_5469a42d428e.slice/crio-a2cd9f614d33bd072ac2687013e35296dcc4972091f19b356b794858613dc65d WatchSource:0}: Error finding container a2cd9f614d33bd072ac2687013e35296dcc4972091f19b356b794858613dc65d: Status 404 returned error can't find the container with id a2cd9f614d33bd072ac2687013e35296dcc4972091f19b356b794858613dc65d
Sep 18 18:05:29.475160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:29.475121    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt" event={"ID":"93782abd-481e-457f-b67c-5469a42d428e","Type":"ContainerStarted","Data":"a2cd9f614d33bd072ac2687013e35296dcc4972091f19b356b794858613dc65d"}
Sep 18 18:05:31.397090 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:31.397034    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:05:31.397441 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:31.397133    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:05:31.489379 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:31.489346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt" event={"ID":"93782abd-481e-457f-b67c-5469a42d428e","Type":"ContainerStarted","Data":"29ac5889802009e6477e93baf7db62c625cc22dd8ffeb9ec314a43753ce613e3"}
Sep 18 18:05:31.489538 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:31.489429    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:31.613643 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:31.613586    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt" podStartSLOduration=1.170997062 podStartE2EDuration="3.613569947s" totalImagesPullingTime="2.442572885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:31.611445519 +0000 UTC m=+315.038658286" watchObservedRunningTime="2026-09-18 18:05:31.613569947 +0000 UTC m=+315.040782701"
Sep 18 18:05:32.497866 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:32.497824    2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-cnvjt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:05:32.498351 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:32.497881    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt" podUID="93782abd-481e-457f-b67c-5469a42d428e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:05:35.497396 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:35.497365    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cnvjt"
Sep 18 18:05:36.460641 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:36.460612    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rchwf"
Sep 18 18:05:37.464295 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:05:37.464259    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:06:26.173384 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.173292    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"]
Sep 18 18:06:26.178769 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.178750    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.181343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.181324    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6c97j\""
Sep 18 18:06:26.182976 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.182955    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 18:06:26.184038 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.184017    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 18:06:26.186800 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.186783    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"]
Sep 18 18:06:26.269846 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.269809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.270034 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.269849    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.270034 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.269923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfqg8\" (UniqueName: \"kubernetes.io/projected/31eb3643-7af6-4048-8326-4d5a1322ae6a-kube-api-access-hfqg8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.273575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.273546    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"]
Sep 18 18:06:26.276238 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.276215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.286660 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.286637    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"]
Sep 18 18:06:26.370570 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.370531    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfqg8\" (UniqueName: \"kubernetes.io/projected/31eb3643-7af6-4048-8326-4d5a1322ae6a-kube-api-access-hfqg8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.370570 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.370585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.370834 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.370642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99gb8\" (UniqueName: \"kubernetes.io/projected/d32334dc-5afd-4b38-8cc4-15820c9c0c68-kube-api-access-99gb8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.370834 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.370676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.370834 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.370760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.370834 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.370785    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.371316 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.371290    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.371438 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.371418    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.373008 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.372987    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"]
Sep 18 18:06:26.375735 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.375718    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.385100 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.385071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfqg8\" (UniqueName: \"kubernetes.io/projected/31eb3643-7af6-4048-8326-4d5a1322ae6a-kube-api-access-hfqg8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.387588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.387566    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"]
Sep 18 18:06:26.472239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.472239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.472239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.472502 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99gb8\" (UniqueName: \"kubernetes.io/projected/d32334dc-5afd-4b38-8cc4-15820c9c0c68-kube-api-access-99gb8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.472502 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472289    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.472502 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472326    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf8tv\" (UniqueName: \"kubernetes.io/projected/b2722df3-07ca-4068-bebf-b861372e9b99-kube-api-access-tf8tv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.472639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472586    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.472639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.472589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.477084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.477043    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"]
Sep 18 18:06:26.479761 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.479747    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.485956 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.485937    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99gb8\" (UniqueName: \"kubernetes.io/projected/d32334dc-5afd-4b38-8cc4-15820c9c0c68-kube-api-access-99gb8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.488685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.488658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:26.491755 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.491732    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"]
Sep 18 18:06:26.573646 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.573580    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96k44\" (UniqueName: \"kubernetes.io/projected/8043e6d9-3617-4213-924a-3428bdeb598f-kube-api-access-96k44\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.573811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.573652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.573811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.573735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.573811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.573801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf8tv\" (UniqueName: \"kubernetes.io/projected/b2722df3-07ca-4068-bebf-b861372e9b99-kube-api-access-tf8tv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.573983 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.573877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.573983 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.573905    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.574418 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.574344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.574522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.574413    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.585672 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.585620    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf8tv\" (UniqueName: \"kubernetes.io/projected/b2722df3-07ca-4068-bebf-b861372e9b99-kube-api-access-tf8tv\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.585672 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.586483    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:26.674895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.674861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k44\" (UniqueName: \"kubernetes.io/projected/8043e6d9-3617-4213-924a-3428bdeb598f-kube-api-access-96k44\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.675063 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.674942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.675063 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.674994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.675519 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.675497    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.675559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.675496    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.683975 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.683951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96k44\" (UniqueName: \"kubernetes.io/projected/8043e6d9-3617-4213-924a-3428bdeb598f-kube-api-access-96k44\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.684681 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.684665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:26.771962 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.771936    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"]
Sep 18 18:06:26.774146 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:26.774116    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31eb3643_7af6_4048_8326_4d5a1322ae6a.slice/crio-4109ea6cb799d538f663a20483fce77b40bf5c9650c990e38262cc54fe6a5627 WatchSource:0}: Error finding container 4109ea6cb799d538f663a20483fce77b40bf5c9650c990e38262cc54fe6a5627: Status 404 returned error can't find the container with id 4109ea6cb799d538f663a20483fce77b40bf5c9650c990e38262cc54fe6a5627
Sep 18 18:06:26.813592 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.813567    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:26.873738 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.871924    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"]
Sep 18 18:06:26.874354 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:26.874325    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32334dc_5afd_4b38_8cc4_15820c9c0c68.slice/crio-1e41538e3662bb7895e10a25ee498f2ba67e9ec8389ad43fa89095f8c92fe0f4 WatchSource:0}: Error finding container 1e41538e3662bb7895e10a25ee498f2ba67e9ec8389ad43fa89095f8c92fe0f4: Status 404 returned error can't find the container with id 1e41538e3662bb7895e10a25ee498f2ba67e9ec8389ad43fa89095f8c92fe0f4
Sep 18 18:06:26.958785 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:26.958717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"]
Sep 18 18:06:26.962234 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:26.962198    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2722df3_07ca_4068_bebf_b861372e9b99.slice/crio-e50474079874f5e25a089f70ce99450180af0eac2a53f8a5960eb201b54a0065 WatchSource:0}: Error finding container e50474079874f5e25a089f70ce99450180af0eac2a53f8a5960eb201b54a0065: Status 404 returned error can't find the container with id e50474079874f5e25a089f70ce99450180af0eac2a53f8a5960eb201b54a0065
Sep 18 18:06:27.095605 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.095580    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"]
Sep 18 18:06:27.128832 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:27.128796    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8043e6d9_3617_4213_924a_3428bdeb598f.slice/crio-74d4e1d691692d1b31a2438b595a5beae0676135b47e05918312d2aff2a18f3d WatchSource:0}: Error finding container 74d4e1d691692d1b31a2438b595a5beae0676135b47e05918312d2aff2a18f3d: Status 404 returned error can't find the container with id 74d4e1d691692d1b31a2438b595a5beae0676135b47e05918312d2aff2a18f3d
Sep 18 18:06:27.771512 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.771475    2582 generic.go:356] "Generic (PLEG): container finished" podID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerID="3e956674fb5e0fac2bcc584e628cf51736813506b000a8c99d5faccc47f4245b" exitCode=0
Sep 18 18:06:27.771991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.771567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm" event={"ID":"31eb3643-7af6-4048-8326-4d5a1322ae6a","Type":"ContainerDied","Data":"3e956674fb5e0fac2bcc584e628cf51736813506b000a8c99d5faccc47f4245b"}
Sep 18 18:06:27.771991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.771630    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm" event={"ID":"31eb3643-7af6-4048-8326-4d5a1322ae6a","Type":"ContainerStarted","Data":"4109ea6cb799d538f663a20483fce77b40bf5c9650c990e38262cc54fe6a5627"}
Sep 18 18:06:27.773563 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.773434    2582 generic.go:356] "Generic (PLEG): container finished" podID="b2722df3-07ca-4068-bebf-b861372e9b99" containerID="bc0b26117ef75c2dee327db2de2d321082f1f61968404fd3ab1fa64888d58e00" exitCode=0
Sep 18 18:06:27.773563 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.773470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt" event={"ID":"b2722df3-07ca-4068-bebf-b861372e9b99","Type":"ContainerDied","Data":"bc0b26117ef75c2dee327db2de2d321082f1f61968404fd3ab1fa64888d58e00"}
Sep 18 18:06:27.773563 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.773518    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt" event={"ID":"b2722df3-07ca-4068-bebf-b861372e9b99","Type":"ContainerStarted","Data":"e50474079874f5e25a089f70ce99450180af0eac2a53f8a5960eb201b54a0065"}
Sep 18 18:06:27.775435 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.775409    2582 generic.go:356] "Generic (PLEG): container finished" podID="8043e6d9-3617-4213-924a-3428bdeb598f" containerID="e05a9157f2b9a36d686bea640b7dd2283e5aff3656ad7a5af0b47415cd1a80ff" exitCode=0
Sep 18 18:06:27.775670 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.775439    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp" event={"ID":"8043e6d9-3617-4213-924a-3428bdeb598f","Type":"ContainerDied","Data":"e05a9157f2b9a36d686bea640b7dd2283e5aff3656ad7a5af0b47415cd1a80ff"}
Sep 18 18:06:27.775670 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.775509    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp" event={"ID":"8043e6d9-3617-4213-924a-3428bdeb598f","Type":"ContainerStarted","Data":"74d4e1d691692d1b31a2438b595a5beae0676135b47e05918312d2aff2a18f3d"}
Sep 18 18:06:27.783945 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.783921    2582 generic.go:356] "Generic (PLEG): container finished" podID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerID="d7e8c2d81ee69a3de3ec2f13835767eb3cf7131709ecb619c132a2b71c43128d" exitCode=0
Sep 18 18:06:27.784063 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.783968    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j" event={"ID":"d32334dc-5afd-4b38-8cc4-15820c9c0c68","Type":"ContainerDied","Data":"d7e8c2d81ee69a3de3ec2f13835767eb3cf7131709ecb619c132a2b71c43128d"}
Sep 18 18:06:27.784125 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:27.784085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j" event={"ID":"d32334dc-5afd-4b38-8cc4-15820c9c0c68","Type":"ContainerStarted","Data":"1e41538e3662bb7895e10a25ee498f2ba67e9ec8389ad43fa89095f8c92fe0f4"}
Sep 18 18:06:29.796130 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.796090    2582 generic.go:356] "Generic (PLEG): container finished" podID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerID="b81af01fb3defb9fd51b7d1dc935684ee2fc3cd4d7148348269f638d86b2b66a" exitCode=0
Sep 18 18:06:29.796525 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.796181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j" event={"ID":"d32334dc-5afd-4b38-8cc4-15820c9c0c68","Type":"ContainerDied","Data":"b81af01fb3defb9fd51b7d1dc935684ee2fc3cd4d7148348269f638d86b2b66a"}
Sep 18 18:06:29.798129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.798107    2582 generic.go:356] "Generic (PLEG): container finished" podID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerID="8e7d001e1660b7bc9a86781f033d8519a6ba2541fa2d528f976529790adcbcc9" exitCode=0
Sep 18 18:06:29.798229 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.798193    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm" event={"ID":"31eb3643-7af6-4048-8326-4d5a1322ae6a","Type":"ContainerDied","Data":"8e7d001e1660b7bc9a86781f033d8519a6ba2541fa2d528f976529790adcbcc9"}
Sep 18 18:06:29.800335 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.800314    2582 generic.go:356] "Generic (PLEG): container finished" podID="b2722df3-07ca-4068-bebf-b861372e9b99" containerID="cebf2f6ae3f837d98cb8308ee89fc2b0ed4d56989dfbecd093ac3acaae996f48" exitCode=0
Sep 18 18:06:29.800416 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.800385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt" event={"ID":"b2722df3-07ca-4068-bebf-b861372e9b99","Type":"ContainerDied","Data":"cebf2f6ae3f837d98cb8308ee89fc2b0ed4d56989dfbecd093ac3acaae996f48"}
Sep 18 18:06:29.802415 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.802396    2582 generic.go:356] "Generic (PLEG): container finished" podID="8043e6d9-3617-4213-924a-3428bdeb598f" containerID="af6c9c99b12b86fdfcb2e1adbe22d127129aaec49ad7f06f624e85a176a9fc4f" exitCode=0
Sep 18 18:06:29.802498 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:29.802449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp" event={"ID":"8043e6d9-3617-4213-924a-3428bdeb598f","Type":"ContainerDied","Data":"af6c9c99b12b86fdfcb2e1adbe22d127129aaec49ad7f06f624e85a176a9fc4f"}
Sep 18 18:06:30.815163 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.815125    2582 generic.go:356] "Generic (PLEG): container finished" podID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerID="9c276afaa2cef77df5389f7ffe5520207c5d6dcc886cfa608404c0e8aa5b68ae" exitCode=0
Sep 18 18:06:30.815600 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.815211    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j" event={"ID":"d32334dc-5afd-4b38-8cc4-15820c9c0c68","Type":"ContainerDied","Data":"9c276afaa2cef77df5389f7ffe5520207c5d6dcc886cfa608404c0e8aa5b68ae"}
Sep 18 18:06:30.817283 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.817262    2582 generic.go:356] "Generic (PLEG): container finished" podID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerID="fb3af2bf79cd6ca4e4538e560ca90ae56fbb453794cc6aaa291a14dfd4866ca9" exitCode=0
Sep 18 18:06:30.817389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.817347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm" event={"ID":"31eb3643-7af6-4048-8326-4d5a1322ae6a","Type":"ContainerDied","Data":"fb3af2bf79cd6ca4e4538e560ca90ae56fbb453794cc6aaa291a14dfd4866ca9"}
Sep 18 18:06:30.819452 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.819431    2582 generic.go:356] "Generic (PLEG): container finished" podID="b2722df3-07ca-4068-bebf-b861372e9b99" containerID="5e1200a882b6cfba20677f443b0a626780db4b3867b1354ab1435151c4d03087" exitCode=0
Sep 18 18:06:30.819561 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.819515    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt" event={"ID":"b2722df3-07ca-4068-bebf-b861372e9b99","Type":"ContainerDied","Data":"5e1200a882b6cfba20677f443b0a626780db4b3867b1354ab1435151c4d03087"}
Sep 18 18:06:30.821599 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.821578    2582 generic.go:356] "Generic (PLEG): container finished" podID="8043e6d9-3617-4213-924a-3428bdeb598f" containerID="afb61afc086b46da91f06ba238ec847a7045f22ef79183ef62183523e78e07b3" exitCode=0
Sep 18 18:06:30.821695 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:30.821640    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp" event={"ID":"8043e6d9-3617-4213-924a-3428bdeb598f","Type":"ContainerDied","Data":"afb61afc086b46da91f06ba238ec847a7045f22ef79183ef62183523e78e07b3"}
Sep 18 18:06:31.964356 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:31.964335    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:32.021439 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.021411    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:32.041839 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.041682    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:32.045682 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.045661    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:32.124134 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf8tv\" (UniqueName: \"kubernetes.io/projected/b2722df3-07ca-4068-bebf-b861372e9b99-kube-api-access-tf8tv\") pod \"b2722df3-07ca-4068-bebf-b861372e9b99\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") "
Sep 18 18:06:32.124134 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124117    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-bundle\") pod \"b2722df3-07ca-4068-bebf-b861372e9b99\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") "
Sep 18 18:06:32.124329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124156    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-bundle\") pod \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") "
Sep 18 18:06:32.124329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124181    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-util\") pod \"b2722df3-07ca-4068-bebf-b861372e9b99\" (UID: \"b2722df3-07ca-4068-bebf-b861372e9b99\") "
Sep 18 18:06:32.124329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124294    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99gb8\" (UniqueName: \"kubernetes.io/projected/d32334dc-5afd-4b38-8cc4-15820c9c0c68-kube-api-access-99gb8\") pod \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") "
Sep 18 18:06:32.124329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124320    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-util\") pod \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\" (UID: \"d32334dc-5afd-4b38-8cc4-15820c9c0c68\") "
Sep 18 18:06:32.124730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.124701    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-bundle" (OuterVolumeSpecName: "bundle") pod "b2722df3-07ca-4068-bebf-b861372e9b99" (UID: "b2722df3-07ca-4068-bebf-b861372e9b99"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.125210 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.125183    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-bundle" (OuterVolumeSpecName: "bundle") pod "d32334dc-5afd-4b38-8cc4-15820c9c0c68" (UID: "d32334dc-5afd-4b38-8cc4-15820c9c0c68"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.126461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.126441    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2722df3-07ca-4068-bebf-b861372e9b99-kube-api-access-tf8tv" (OuterVolumeSpecName: "kube-api-access-tf8tv") pod "b2722df3-07ca-4068-bebf-b861372e9b99" (UID: "b2722df3-07ca-4068-bebf-b861372e9b99"). InnerVolumeSpecName "kube-api-access-tf8tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:32.126879 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.126855    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d32334dc-5afd-4b38-8cc4-15820c9c0c68-kube-api-access-99gb8" (OuterVolumeSpecName: "kube-api-access-99gb8") pod "d32334dc-5afd-4b38-8cc4-15820c9c0c68" (UID: "d32334dc-5afd-4b38-8cc4-15820c9c0c68"). InnerVolumeSpecName "kube-api-access-99gb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:32.130061 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.130031    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-util" (OuterVolumeSpecName: "util") pod "d32334dc-5afd-4b38-8cc4-15820c9c0c68" (UID: "d32334dc-5afd-4b38-8cc4-15820c9c0c68"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.130749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.130731    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-util" (OuterVolumeSpecName: "util") pod "b2722df3-07ca-4068-bebf-b861372e9b99" (UID: "b2722df3-07ca-4068-bebf-b861372e9b99"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.225481 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225434    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-bundle\") pod \"8043e6d9-3617-4213-924a-3428bdeb598f\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") "
Sep 18 18:06:32.225655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225499    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-util\") pod \"31eb3643-7af6-4048-8326-4d5a1322ae6a\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") "
Sep 18 18:06:32.225655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225526    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfqg8\" (UniqueName: \"kubernetes.io/projected/31eb3643-7af6-4048-8326-4d5a1322ae6a-kube-api-access-hfqg8\") pod \"31eb3643-7af6-4048-8326-4d5a1322ae6a\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") "
Sep 18 18:06:32.225655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225543    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96k44\" (UniqueName: \"kubernetes.io/projected/8043e6d9-3617-4213-924a-3428bdeb598f-kube-api-access-96k44\") pod \"8043e6d9-3617-4213-924a-3428bdeb598f\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") "
Sep 18 18:06:32.225655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225565    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-bundle\") pod \"31eb3643-7af6-4048-8326-4d5a1322ae6a\" (UID: \"31eb3643-7af6-4048-8326-4d5a1322ae6a\") "
Sep 18 18:06:32.225986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225658    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-util\") pod \"8043e6d9-3617-4213-924a-3428bdeb598f\" (UID: \"8043e6d9-3617-4213-924a-3428bdeb598f\") "
Sep 18 18:06:32.225986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225929    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99gb8\" (UniqueName: \"kubernetes.io/projected/d32334dc-5afd-4b38-8cc4-15820c9c0c68-kube-api-access-99gb8\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.225986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225948    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.225986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225964    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf8tv\" (UniqueName: \"kubernetes.io/projected/b2722df3-07ca-4068-bebf-b861372e9b99-kube-api-access-tf8tv\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.225986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225979    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.226266 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.225992    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d32334dc-5afd-4b38-8cc4-15820c9c0c68-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.226266 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.226004    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2722df3-07ca-4068-bebf-b861372e9b99-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.226266 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.226087    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-bundle" (OuterVolumeSpecName: "bundle") pod "8043e6d9-3617-4213-924a-3428bdeb598f" (UID: "8043e6d9-3617-4213-924a-3428bdeb598f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.226266 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.226195    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-bundle" (OuterVolumeSpecName: "bundle") pod "31eb3643-7af6-4048-8326-4d5a1322ae6a" (UID: "31eb3643-7af6-4048-8326-4d5a1322ae6a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.228379 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.228346    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31eb3643-7af6-4048-8326-4d5a1322ae6a-kube-api-access-hfqg8" (OuterVolumeSpecName: "kube-api-access-hfqg8") pod "31eb3643-7af6-4048-8326-4d5a1322ae6a" (UID: "31eb3643-7af6-4048-8326-4d5a1322ae6a"). InnerVolumeSpecName "kube-api-access-hfqg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:32.228468 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.228372    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8043e6d9-3617-4213-924a-3428bdeb598f-kube-api-access-96k44" (OuterVolumeSpecName: "kube-api-access-96k44") pod "8043e6d9-3617-4213-924a-3428bdeb598f" (UID: "8043e6d9-3617-4213-924a-3428bdeb598f"). InnerVolumeSpecName "kube-api-access-96k44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:32.231039 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.231009    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-util" (OuterVolumeSpecName: "util") pod "31eb3643-7af6-4048-8326-4d5a1322ae6a" (UID: "31eb3643-7af6-4048-8326-4d5a1322ae6a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.231575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.231555    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-util" (OuterVolumeSpecName: "util") pod "8043e6d9-3617-4213-924a-3428bdeb598f" (UID: "8043e6d9-3617-4213-924a-3428bdeb598f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:32.327278 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.327247    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.327278 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.327271    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.327278 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.327280    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfqg8\" (UniqueName: \"kubernetes.io/projected/31eb3643-7af6-4048-8326-4d5a1322ae6a-kube-api-access-hfqg8\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.327500 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.327290    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96k44\" (UniqueName: \"kubernetes.io/projected/8043e6d9-3617-4213-924a-3428bdeb598f-kube-api-access-96k44\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.327500 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.327300    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31eb3643-7af6-4048-8326-4d5a1322ae6a-bundle\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.327500 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.327308    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8043e6d9-3617-4213-924a-3428bdeb598f-util\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:06:32.834811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.834775    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp"
Sep 18 18:06:32.834811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.834800    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b6wqp" event={"ID":"8043e6d9-3617-4213-924a-3428bdeb598f","Type":"ContainerDied","Data":"74d4e1d691692d1b31a2438b595a5beae0676135b47e05918312d2aff2a18f3d"}
Sep 18 18:06:32.835025 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.834858    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74d4e1d691692d1b31a2438b595a5beae0676135b47e05918312d2aff2a18f3d"
Sep 18 18:06:32.836808 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.836792    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j"
Sep 18 18:06:32.836931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.836804    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abjx4j" event={"ID":"d32334dc-5afd-4b38-8cc4-15820c9c0c68","Type":"ContainerDied","Data":"1e41538e3662bb7895e10a25ee498f2ba67e9ec8389ad43fa89095f8c92fe0f4"}
Sep 18 18:06:32.836931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.836842    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e41538e3662bb7895e10a25ee498f2ba67e9ec8389ad43fa89095f8c92fe0f4"
Sep 18 18:06:32.838943 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.838921    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm"
Sep 18 18:06:32.839024 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.838972    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cg7wrm" event={"ID":"31eb3643-7af6-4048-8326-4d5a1322ae6a","Type":"ContainerDied","Data":"4109ea6cb799d538f663a20483fce77b40bf5c9650c990e38262cc54fe6a5627"}
Sep 18 18:06:32.839024 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.839014    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4109ea6cb799d538f663a20483fce77b40bf5c9650c990e38262cc54fe6a5627"
Sep 18 18:06:32.841040 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.841010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt" event={"ID":"b2722df3-07ca-4068-bebf-b861372e9b99","Type":"ContainerDied","Data":"e50474079874f5e25a089f70ce99450180af0eac2a53f8a5960eb201b54a0065"}
Sep 18 18:06:32.841145 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.841039    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nmqkt"
Sep 18 18:06:32.841145 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:32.841073    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e50474079874f5e25a089f70ce99450180af0eac2a53f8a5960eb201b54a0065"
Sep 18 18:06:38.358211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358179    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"]
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358513    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="extract"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358524    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="extract"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358534    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="util"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358539    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="util"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358546    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="pull"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358552    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="pull"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358560    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="pull"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358565    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="pull"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358573    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="extract"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358579    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="extract"
Sep 18 18:06:38.358584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358587    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="util"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358592    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="util"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358601    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="util"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358605    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="util"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358613    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="util"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358618    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="util"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358627    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358632    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358637    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358642    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358651    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="pull"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358655    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="pull"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358662    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="pull"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358667    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="pull"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358721    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8043e6d9-3617-4213-924a-3428bdeb598f" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358728    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d32334dc-5afd-4b38-8cc4-15820c9c0c68" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358733    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2722df3-07ca-4068-bebf-b861372e9b99" containerName="extract"
Sep 18 18:06:38.358915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.358741    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="31eb3643-7af6-4048-8326-4d5a1322ae6a" containerName="extract"
Sep 18 18:06:38.363473 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.363456    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:06:38.368239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.368204    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 18:06:38.368239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.368203    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 18:06:38.368447 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.368243    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rrxp7\""
Sep 18 18:06:38.369072 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.369029    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 18 18:06:38.376106 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.376083    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"]
Sep 18 18:06:38.381901 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.381880    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckc9h\" (UniqueName: \"kubernetes.io/projected/f111d43f-fb09-4686-b4d8-1277a721ba1d-kube-api-access-ckc9h\") pod \"dns-operator-controller-manager-7b84cd7c9c-6pnvq\" (UID: \"f111d43f-fb09-4686-b4d8-1277a721ba1d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:06:38.482869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.482834    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckc9h\" (UniqueName: \"kubernetes.io/projected/f111d43f-fb09-4686-b4d8-1277a721ba1d-kube-api-access-ckc9h\") pod \"dns-operator-controller-manager-7b84cd7c9c-6pnvq\" (UID: \"f111d43f-fb09-4686-b4d8-1277a721ba1d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:06:38.498504 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.498471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckc9h\" (UniqueName: \"kubernetes.io/projected/f111d43f-fb09-4686-b4d8-1277a721ba1d-kube-api-access-ckc9h\") pod \"dns-operator-controller-manager-7b84cd7c9c-6pnvq\" (UID: \"f111d43f-fb09-4686-b4d8-1277a721ba1d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:06:38.673752 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.673659    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:06:38.961322 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:38.961299    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"]
Sep 18 18:06:38.962944 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:38.962918    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf111d43f_fb09_4686_b4d8_1277a721ba1d.slice/crio-be3032e4113015319df046d751e8d2da94536e8e7ba6fc7374e3722c42047ef9 WatchSource:0}: Error finding container be3032e4113015319df046d751e8d2da94536e8e7ba6fc7374e3722c42047ef9: Status 404 returned error can't find the container with id be3032e4113015319df046d751e8d2da94536e8e7ba6fc7374e3722c42047ef9
Sep 18 18:06:39.246369 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.246274    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c88668489-fngp8"]
Sep 18 18:06:39.251657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.251630    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.255360 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.255333    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 18:06:39.255739 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.255718    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 18:06:39.256547 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.256508    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 18:06:39.256805 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.256577    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 18:06:39.256992 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.256603    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 18:06:39.259211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.258817    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 18:06:39.259211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.258874    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ffzz9\""
Sep 18 18:06:39.259211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.259025    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 18:06:39.263213 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.263190    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 18:06:39.274667 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.274644    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c88668489-fngp8"]
Sep 18 18:06:39.290321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290295    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-oauth-serving-cert\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.290404 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkfx\" (UniqueName: \"kubernetes.io/projected/df945b42-5456-4a33-85c8-420ab4bf72ad-kube-api-access-vjkfx\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.290456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290415    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df945b42-5456-4a33-85c8-420ab4bf72ad-console-serving-cert\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.290510 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-trusted-ca-bundle\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.290559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290541    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-service-ca\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.290624 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df945b42-5456-4a33-85c8-420ab4bf72ad-console-oauth-config\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.290663 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.290631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-console-config\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.391831 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-trusted-ca-bundle\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.391831 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-service-ca\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df945b42-5456-4a33-85c8-420ab4bf72ad-console-oauth-config\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-console-config\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391905    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-oauth-serving-cert\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391929    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkfx\" (UniqueName: \"kubernetes.io/projected/df945b42-5456-4a33-85c8-420ab4bf72ad-kube-api-access-vjkfx\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.391953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df945b42-5456-4a33-85c8-420ab4bf72ad-console-serving-cert\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392787 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.392768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-console-config\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392871 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.392773    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-oauth-serving-cert\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.392914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.392873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-service-ca\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.393180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.393159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df945b42-5456-4a33-85c8-420ab4bf72ad-trusted-ca-bundle\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.394537 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.394520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df945b42-5456-4a33-85c8-420ab4bf72ad-console-oauth-config\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.394648 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.394632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df945b42-5456-4a33-85c8-420ab4bf72ad-console-serving-cert\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.401493 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.401468    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkfx\" (UniqueName: \"kubernetes.io/projected/df945b42-5456-4a33-85c8-420ab4bf72ad-kube-api-access-vjkfx\") pod \"console-7c88668489-fngp8\" (UID: \"df945b42-5456-4a33-85c8-420ab4bf72ad\") " pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.564801 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.564767    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:39.853113 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.853080    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c88668489-fngp8"]
Sep 18 18:06:39.856897 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:39.856859    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf945b42_5456_4a33_85c8_420ab4bf72ad.slice/crio-09a988b852ba945bd0d77230ebe0bff5708c2b1d294d99ae4963a4a3a5c19d79 WatchSource:0}: Error finding container 09a988b852ba945bd0d77230ebe0bff5708c2b1d294d99ae4963a4a3a5c19d79: Status 404 returned error can't find the container with id 09a988b852ba945bd0d77230ebe0bff5708c2b1d294d99ae4963a4a3a5c19d79
Sep 18 18:06:39.889721 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.889688    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c88668489-fngp8" event={"ID":"df945b42-5456-4a33-85c8-420ab4bf72ad","Type":"ContainerStarted","Data":"09a988b852ba945bd0d77230ebe0bff5708c2b1d294d99ae4963a4a3a5c19d79"}
Sep 18 18:06:39.892409 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:39.892328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq" event={"ID":"f111d43f-fb09-4686-b4d8-1277a721ba1d","Type":"ContainerStarted","Data":"be3032e4113015319df046d751e8d2da94536e8e7ba6fc7374e3722c42047ef9"}
Sep 18 18:06:40.900881 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:40.900841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c88668489-fngp8" event={"ID":"df945b42-5456-4a33-85c8-420ab4bf72ad","Type":"ContainerStarted","Data":"2f9c3538a4d80140c1b91c02f058f6ecf5bc6a1c133ef0f8da41057551dd9b5d"}
Sep 18 18:06:41.908394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:41.908354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq" event={"ID":"f111d43f-fb09-4686-b4d8-1277a721ba1d","Type":"ContainerStarted","Data":"55d3d209dc5c3eb73f641d95a6d3a96d9ebc8cf094385b2419046b5667007fea"}
Sep 18 18:06:41.908817 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:41.908595    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:06:41.931133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:41.931007    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c88668489-fngp8" podStartSLOduration=2.930988966 podStartE2EDuration="2.930988966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:06:40.938929738 +0000 UTC m=+384.366142494" watchObservedRunningTime="2026-09-18 18:06:41.930988966 +0000 UTC m=+385.358201721"
Sep 18 18:06:41.932853 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:41.932816    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq" podStartSLOduration=1.226310506 podStartE2EDuration="3.932805443s" totalImagesPullingTime="2.706494937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:06:41.93007229 +0000 UTC m=+385.357285036" watchObservedRunningTime="2026-09-18 18:06:41.932805443 +0000 UTC m=+385.360018198"
Sep 18 18:06:45.371918 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.371885    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-v4qkj"]
Sep 18 18:06:45.375368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.375351    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:06:45.378487 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.378466    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sh6xm\""
Sep 18 18:06:45.444929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.444869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4twdh\" (UniqueName: \"kubernetes.io/projected/60d3af17-9608-4189-a31c-39e85ebf883f-kube-api-access-4twdh\") pod \"authorino-operator-76b478bd78-v4qkj\" (UID: \"60d3af17-9608-4189-a31c-39e85ebf883f\") " pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:06:45.460739 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.460705    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-v4qkj"]
Sep 18 18:06:45.545678 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.545648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4twdh\" (UniqueName: \"kubernetes.io/projected/60d3af17-9608-4189-a31c-39e85ebf883f-kube-api-access-4twdh\") pod \"authorino-operator-76b478bd78-v4qkj\" (UID: \"60d3af17-9608-4189-a31c-39e85ebf883f\") " pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:06:45.562999 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.562966    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4twdh\" (UniqueName: \"kubernetes.io/projected/60d3af17-9608-4189-a31c-39e85ebf883f-kube-api-access-4twdh\") pod \"authorino-operator-76b478bd78-v4qkj\" (UID: \"60d3af17-9608-4189-a31c-39e85ebf883f\") " pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:06:45.685515 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.685422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:06:45.961612 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:45.961587    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-v4qkj"]
Sep 18 18:06:45.963441 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:06:45.963416    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d3af17_9608_4189_a31c_39e85ebf883f.slice/crio-8cd4e10996ddcc8a844deab9dc214b29a45a0eeab698f77c348cd2190f3d9325 WatchSource:0}: Error finding container 8cd4e10996ddcc8a844deab9dc214b29a45a0eeab698f77c348cd2190f3d9325: Status 404 returned error can't find the container with id 8cd4e10996ddcc8a844deab9dc214b29a45a0eeab698f77c348cd2190f3d9325
Sep 18 18:06:46.934681 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:46.934649    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj" event={"ID":"60d3af17-9608-4189-a31c-39e85ebf883f","Type":"ContainerStarted","Data":"8cd4e10996ddcc8a844deab9dc214b29a45a0eeab698f77c348cd2190f3d9325"}
Sep 18 18:06:49.565672 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.565640    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:49.566084 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.565925    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:49.570850 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.570828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:49.953733 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.953636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj" event={"ID":"60d3af17-9608-4189-a31c-39e85ebf883f","Type":"ContainerStarted","Data":"882311e7dc1d0b937bc8ca527a896c6706053a56f6e6736f8ebf1e1fdb1f5362"}
Sep 18 18:06:49.953882 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.953786    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:06:49.957657 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.957634    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c88668489-fngp8"
Sep 18 18:06:49.975448 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:49.975394    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj" podStartSLOduration=1.851425449 podStartE2EDuration="4.975375765s" totalImagesPullingTime="3.123950316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:06:49.973658229 +0000 UTC m=+393.400870983" watchObservedRunningTime="2026-09-18 18:06:49.975375765 +0000 UTC m=+393.402588520"
Sep 18 18:06:52.919679 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:06:52.919652    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-6pnvq"
Sep 18 18:07:00.961510 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:00.961480    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-v4qkj"
Sep 18 18:07:01.636673 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.636638    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"]
Sep 18 18:07:01.640237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.640212    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.644660 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.644635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j6zlm\""
Sep 18 18:07:01.670486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.670455    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"]
Sep 18 18:07:01.794157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.794112    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2648190d-b1f3-42f2-b240-77b794343278-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qss94\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.794157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.794161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dn2\" (UniqueName: \"kubernetes.io/projected/2648190d-b1f3-42f2-b240-77b794343278-kube-api-access-b8dn2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qss94\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.895371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.895278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2648190d-b1f3-42f2-b240-77b794343278-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qss94\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.895371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.895333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dn2\" (UniqueName: \"kubernetes.io/projected/2648190d-b1f3-42f2-b240-77b794343278-kube-api-access-b8dn2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qss94\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.895694 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.895675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2648190d-b1f3-42f2-b240-77b794343278-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qss94\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.921486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.921451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dn2\" (UniqueName: \"kubernetes.io/projected/2648190d-b1f3-42f2-b240-77b794343278-kube-api-access-b8dn2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qss94\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:01.951376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:01.951339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:02.244069 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:02.244020    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"]
Sep 18 18:07:02.245445 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:07:02.245416    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2648190d_b1f3_42f2_b240_77b794343278.slice/crio-b5cb1dc2a57cad51eaf84ace842fda0c958f3155ce3bc6b38fabb0451860bbfb WatchSource:0}: Error finding container b5cb1dc2a57cad51eaf84ace842fda0c958f3155ce3bc6b38fabb0451860bbfb: Status 404 returned error can't find the container with id b5cb1dc2a57cad51eaf84ace842fda0c958f3155ce3bc6b38fabb0451860bbfb
Sep 18 18:07:03.028441 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:03.028405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94" event={"ID":"2648190d-b1f3-42f2-b240-77b794343278","Type":"ContainerStarted","Data":"b5cb1dc2a57cad51eaf84ace842fda0c958f3155ce3bc6b38fabb0451860bbfb"}
Sep 18 18:07:06.048583 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:06.048551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94" event={"ID":"2648190d-b1f3-42f2-b240-77b794343278","Type":"ContainerStarted","Data":"84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2"}
Sep 18 18:07:06.048888 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:06.048647    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:06.096953 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:06.096911    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94" podStartSLOduration=1.38993151 podStartE2EDuration="5.096896379s" totalImagesPullingTime="3.706964869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:06.094212786 +0000 UTC m=+409.521425541" watchObservedRunningTime="2026-09-18 18:07:06.096896379 +0000 UTC m=+409.524109131"
Sep 18 18:07:14.593250 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:14.593216    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"]
Sep 18 18:07:14.593637 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:14.593414    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94" podUID="2648190d-b1f3-42f2-b240-77b794343278" containerName="manager" containerID="cri-o://84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2" gracePeriod=10
Sep 18 18:07:14.595281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:14.595250    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:14.840408 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:14.840383    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:15.010464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.010389    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2648190d-b1f3-42f2-b240-77b794343278-extensions-socket-volume\") pod \"2648190d-b1f3-42f2-b240-77b794343278\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") "
Sep 18 18:07:15.010618 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.010491    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8dn2\" (UniqueName: \"kubernetes.io/projected/2648190d-b1f3-42f2-b240-77b794343278-kube-api-access-b8dn2\") pod \"2648190d-b1f3-42f2-b240-77b794343278\" (UID: \"2648190d-b1f3-42f2-b240-77b794343278\") "
Sep 18 18:07:15.011005 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.010826    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2648190d-b1f3-42f2-b240-77b794343278-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2648190d-b1f3-42f2-b240-77b794343278" (UID: "2648190d-b1f3-42f2-b240-77b794343278"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:07:15.012846 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.012815    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2648190d-b1f3-42f2-b240-77b794343278-kube-api-access-b8dn2" (OuterVolumeSpecName: "kube-api-access-b8dn2") pod "2648190d-b1f3-42f2-b240-77b794343278" (UID: "2648190d-b1f3-42f2-b240-77b794343278"). InnerVolumeSpecName "kube-api-access-b8dn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:07:15.099348 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.099311    2582 generic.go:356] "Generic (PLEG): container finished" podID="2648190d-b1f3-42f2-b240-77b794343278" containerID="84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2" exitCode=0
Sep 18 18:07:15.099522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.099358    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"
Sep 18 18:07:15.099522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.099441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94" event={"ID":"2648190d-b1f3-42f2-b240-77b794343278","Type":"ContainerDied","Data":"84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2"}
Sep 18 18:07:15.099522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.099488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94" event={"ID":"2648190d-b1f3-42f2-b240-77b794343278","Type":"ContainerDied","Data":"b5cb1dc2a57cad51eaf84ace842fda0c958f3155ce3bc6b38fabb0451860bbfb"}
Sep 18 18:07:15.099522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.099513    2582 scope.go:117] "RemoveContainer" containerID="84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2"
Sep 18 18:07:15.109610 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.109585    2582 scope.go:117] "RemoveContainer" containerID="84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2"
Sep 18 18:07:15.109893 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:07:15.109868    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2\": container with ID starting with 84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2 not found: ID does not exist" containerID="84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2"
Sep 18 18:07:15.109967 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.109906    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2"} err="failed to get container status \"84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2\": rpc error: code = NotFound desc = could not find container \"84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2\": container with ID starting with 84a050a256380bfa331c85d2e0f862a4e9049807594cec752c34f1c0440589c2 not found: ID does not exist"
Sep 18 18:07:15.111259 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.111242    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8dn2\" (UniqueName: \"kubernetes.io/projected/2648190d-b1f3-42f2-b240-77b794343278-kube-api-access-b8dn2\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.111312 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.111262    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2648190d-b1f3-42f2-b240-77b794343278-extensions-socket-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.127999 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.127974    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"]
Sep 18 18:07:15.138226 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:15.138199    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qss94"]
Sep 18 18:07:16.920440 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:16.920408    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2648190d-b1f3-42f2-b240-77b794343278" path="/var/lib/kubelet/pods/2648190d-b1f3-42f2-b240-77b794343278/volumes"
Sep 18 18:07:30.984487 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:30.984452    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"]
Sep 18 18:07:30.985364 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:30.984819    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2648190d-b1f3-42f2-b240-77b794343278" containerName="manager"
Sep 18 18:07:30.985364 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:30.984830    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2648190d-b1f3-42f2-b240-77b794343278" containerName="manager"
Sep 18 18:07:30.985364 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:30.984889    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2648190d-b1f3-42f2-b240-77b794343278" containerName="manager"
Sep 18 18:07:30.988021 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:30.988002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:30.990528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:30.990505    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4tlzk\""
Sep 18 18:07:31.002851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.002828    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"]
Sep 18 18:07:31.037388 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037393    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037426    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037472    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037591 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037591 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc00339-98d4-4edd-8e78-8ebbae73a641-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037653 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037610    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037697 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.037697 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.037688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shc2l\" (UniqueName: \"kubernetes.io/projected/7cc00339-98d4-4edd-8e78-8ebbae73a641-kube-api-access-shc2l\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138688 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138688 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc00339-98d4-4edd-8e78-8ebbae73a641-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.138929 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.138877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shc2l\" (UniqueName: \"kubernetes.io/projected/7cc00339-98d4-4edd-8e78-8ebbae73a641-kube-api-access-shc2l\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.139307 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.139182    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.139307 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.139237    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.139413 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.139390    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.139502 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.139469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.139963 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.139940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc00339-98d4-4edd-8e78-8ebbae73a641-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.141313 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.141293    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.141630 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.141608    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.149120 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.149073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shc2l\" (UniqueName: \"kubernetes.io/projected/7cc00339-98d4-4edd-8e78-8ebbae73a641-kube-api-access-shc2l\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.149611 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.149590    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc00339-98d4-4edd-8e78-8ebbae73a641-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tvsjk\" (UID: \"7cc00339-98d4-4edd-8e78-8ebbae73a641\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.300461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.300428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:31.583662 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.583635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"]
Sep 18 18:07:31.586157 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:07:31.586124    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc00339_98d4_4edd_8e78_8ebbae73a641.slice/crio-1d76d84a8d9c6606c76565550c370a4d12e228d0fc05aec77eb480c2b7b9d365 WatchSource:0}: Error finding container 1d76d84a8d9c6606c76565550c370a4d12e228d0fc05aec77eb480c2b7b9d365: Status 404 returned error can't find the container with id 1d76d84a8d9c6606c76565550c370a4d12e228d0fc05aec77eb480c2b7b9d365
Sep 18 18:07:31.588388 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.588360    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:07:31.588456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.588430    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:07:31.588496 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:31.588461    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:07:32.200180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:32.200141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" event={"ID":"7cc00339-98d4-4edd-8e78-8ebbae73a641","Type":"ContainerStarted","Data":"6f0c54371012eb2d3517266ab92eb9ed8c4072a599ea62af9a20714cbb9e058d"}
Sep 18 18:07:32.200639 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:32.200208    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" event={"ID":"7cc00339-98d4-4edd-8e78-8ebbae73a641","Type":"ContainerStarted","Data":"1d76d84a8d9c6606c76565550c370a4d12e228d0fc05aec77eb480c2b7b9d365"}
Sep 18 18:07:32.224542 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:32.224499    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podStartSLOduration=2.224484316 podStartE2EDuration="2.224484316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:32.222919289 +0000 UTC m=+435.650132043" watchObservedRunningTime="2026-09-18 18:07:32.224484316 +0000 UTC m=+435.651697156"
Sep 18 18:07:32.300786 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:32.300756    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:32.305479 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:32.305458    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:33.205826 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.205796    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:33.206777 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.206753    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:07:33.270938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.270905    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:33.275119 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.275035    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.277463 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.277418    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 18:07:33.277578 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.277448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j2d5b\""
Sep 18 18:07:33.283714 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.283658    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:33.356366 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.356328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5ljf\" (UniqueName: \"kubernetes.io/projected/bde1c908-e6cb-4b4b-9dac-c8819349d72b-kube-api-access-r5ljf\") pod \"limitador-limitador-8588749977-wt2dp\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") " pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.356554 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.356461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bde1c908-e6cb-4b4b-9dac-c8819349d72b-config-file\") pod \"limitador-limitador-8588749977-wt2dp\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") " pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.369867 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.369830    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:33.457015 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.456942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bde1c908-e6cb-4b4b-9dac-c8819349d72b-config-file\") pod \"limitador-limitador-8588749977-wt2dp\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") " pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.457179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.457145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5ljf\" (UniqueName: \"kubernetes.io/projected/bde1c908-e6cb-4b4b-9dac-c8819349d72b-kube-api-access-r5ljf\") pod \"limitador-limitador-8588749977-wt2dp\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") " pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.457869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.457848    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bde1c908-e6cb-4b4b-9dac-c8819349d72b-config-file\") pod \"limitador-limitador-8588749977-wt2dp\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") " pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.470028 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.469993    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5ljf\" (UniqueName: \"kubernetes.io/projected/bde1c908-e6cb-4b4b-9dac-c8819349d72b-kube-api-access-r5ljf\") pod \"limitador-limitador-8588749977-wt2dp\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") " pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.587667 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.587629    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:33.863466 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:33.863437    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:33.864918 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:07:33.864888    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde1c908_e6cb_4b4b_9dac_c8819349d72b.slice/crio-ef82f1feaf4cc7a9ba5544d4996d0fa7e726727dd05ceb52686f3b5007d3b3db WatchSource:0}: Error finding container ef82f1feaf4cc7a9ba5544d4996d0fa7e726727dd05ceb52686f3b5007d3b3db: Status 404 returned error can't find the container with id ef82f1feaf4cc7a9ba5544d4996d0fa7e726727dd05ceb52686f3b5007d3b3db
Sep 18 18:07:34.216007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:34.215903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp" event={"ID":"bde1c908-e6cb-4b4b-9dac-c8819349d72b","Type":"ContainerStarted","Data":"ef82f1feaf4cc7a9ba5544d4996d0fa7e726727dd05ceb52686f3b5007d3b3db"}
Sep 18 18:07:38.238913 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:38.238815    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp" event={"ID":"bde1c908-e6cb-4b4b-9dac-c8819349d72b","Type":"ContainerStarted","Data":"62d1a98efe69b34ead2c251249a1aaeedf14070cd86ba96b259bf9d56ee25852"}
Sep 18 18:07:38.239282 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:38.238910    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:38.259541 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:38.259496    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp" podStartSLOduration=1.138394123 podStartE2EDuration="5.259482531s" totalImagesPullingTime="4.121088408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:38.25750791 +0000 UTC m=+441.684720656" watchObservedRunningTime="2026-09-18 18:07:38.259482531 +0000 UTC m=+441.686695284"
Sep 18 18:07:48.767940 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:48.767906    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:48.768393 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:48.768097    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp" podUID="bde1c908-e6cb-4b4b-9dac-c8819349d72b" containerName="limitador" containerID="cri-o://62d1a98efe69b34ead2c251249a1aaeedf14070cd86ba96b259bf9d56ee25852" gracePeriod=30
Sep 18 18:07:48.768823 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:48.768770    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:49.296596 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.296564    2582 generic.go:356] "Generic (PLEG): container finished" podID="bde1c908-e6cb-4b4b-9dac-c8819349d72b" containerID="62d1a98efe69b34ead2c251249a1aaeedf14070cd86ba96b259bf9d56ee25852" exitCode=0
Sep 18 18:07:49.296704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.296654    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp" event={"ID":"bde1c908-e6cb-4b4b-9dac-c8819349d72b","Type":"ContainerDied","Data":"62d1a98efe69b34ead2c251249a1aaeedf14070cd86ba96b259bf9d56ee25852"}
Sep 18 18:07:49.296758 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.296748    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp" event={"ID":"bde1c908-e6cb-4b4b-9dac-c8819349d72b","Type":"ContainerDied","Data":"ef82f1feaf4cc7a9ba5544d4996d0fa7e726727dd05ceb52686f3b5007d3b3db"}
Sep 18 18:07:49.296811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.296771    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef82f1feaf4cc7a9ba5544d4996d0fa7e726727dd05ceb52686f3b5007d3b3db"
Sep 18 18:07:49.309646 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.309629    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:49.386967 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.386883    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bde1c908-e6cb-4b4b-9dac-c8819349d72b-config-file\") pod \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") "
Sep 18 18:07:49.387163 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.387007    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5ljf\" (UniqueName: \"kubernetes.io/projected/bde1c908-e6cb-4b4b-9dac-c8819349d72b-kube-api-access-r5ljf\") pod \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\" (UID: \"bde1c908-e6cb-4b4b-9dac-c8819349d72b\") "
Sep 18 18:07:49.387261 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.387235    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde1c908-e6cb-4b4b-9dac-c8819349d72b-config-file" (OuterVolumeSpecName: "config-file") pod "bde1c908-e6cb-4b4b-9dac-c8819349d72b" (UID: "bde1c908-e6cb-4b4b-9dac-c8819349d72b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:07:49.387343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.387327    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bde1c908-e6cb-4b4b-9dac-c8819349d72b-config-file\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:07:49.389273 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.389253    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bde1c908-e6cb-4b4b-9dac-c8819349d72b-kube-api-access-r5ljf" (OuterVolumeSpecName: "kube-api-access-r5ljf") pod "bde1c908-e6cb-4b4b-9dac-c8819349d72b" (UID: "bde1c908-e6cb-4b4b-9dac-c8819349d72b"). InnerVolumeSpecName "kube-api-access-r5ljf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:07:49.488451 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:49.488422    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5ljf\" (UniqueName: \"kubernetes.io/projected/bde1c908-e6cb-4b4b-9dac-c8819349d72b-kube-api-access-r5ljf\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:07:50.301604 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:50.301573    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-wt2dp"
Sep 18 18:07:50.324035 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:50.324005    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:50.329890 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:50.329867    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-wt2dp"]
Sep 18 18:07:50.919557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:50.919523    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bde1c908-e6cb-4b4b-9dac-c8819349d72b" path="/var/lib/kubelet/pods/bde1c908-e6cb-4b4b-9dac-c8819349d72b/volumes"
Sep 18 18:07:54.926664 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.926635    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"]
Sep 18 18:07:54.927129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.927000    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bde1c908-e6cb-4b4b-9dac-c8819349d72b" containerName="limitador"
Sep 18 18:07:54.927129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.927012    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bde1c908-e6cb-4b4b-9dac-c8819349d72b" containerName="limitador"
Sep 18 18:07:54.927129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.927118    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bde1c908-e6cb-4b4b-9dac-c8819349d72b" containerName="limitador"
Sep 18 18:07:54.929424 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.929397    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"]
Sep 18 18:07:54.929568 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.929491    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:54.932858 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.932835    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 18 18:07:54.933592 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.933572    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:07:54.933683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.933596    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-czq8t\""
Sep 18 18:07:54.933946 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:54.933928    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:07:55.036897 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.036864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hbp4\" (UniqueName: \"kubernetes.io/projected/de81cfc0-ae09-4d04-8c06-0213ccc6d3c3-kube-api-access-2hbp4\") pod \"postgres-69fb4594cf-svhjq\" (UID: \"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.037092 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.036914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/de81cfc0-ae09-4d04-8c06-0213ccc6d3c3-data\") pod \"postgres-69fb4594cf-svhjq\" (UID: \"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.137442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.137409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbp4\" (UniqueName: \"kubernetes.io/projected/de81cfc0-ae09-4d04-8c06-0213ccc6d3c3-kube-api-access-2hbp4\") pod \"postgres-69fb4594cf-svhjq\" (UID: \"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.137582 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.137466    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/de81cfc0-ae09-4d04-8c06-0213ccc6d3c3-data\") pod \"postgres-69fb4594cf-svhjq\" (UID: \"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.137810 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.137790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/de81cfc0-ae09-4d04-8c06-0213ccc6d3c3-data\") pod \"postgres-69fb4594cf-svhjq\" (UID: \"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.146577 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.146555    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbp4\" (UniqueName: \"kubernetes.io/projected/de81cfc0-ae09-4d04-8c06-0213ccc6d3c3-kube-api-access-2hbp4\") pod \"postgres-69fb4594cf-svhjq\" (UID: \"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.239733 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.239644    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:07:55.514707 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:55.514675    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"]
Sep 18 18:07:55.516139 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:07:55.516105    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde81cfc0_ae09_4d04_8c06_0213ccc6d3c3.slice/crio-ccddb87fac52d2aed8c39a9106352b3f691aadac78060c565360687ffd645ef6 WatchSource:0}: Error finding container ccddb87fac52d2aed8c39a9106352b3f691aadac78060c565360687ffd645ef6: Status 404 returned error can't find the container with id ccddb87fac52d2aed8c39a9106352b3f691aadac78060c565360687ffd645ef6
Sep 18 18:07:56.336299 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:07:56.336263    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq" event={"ID":"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3","Type":"ContainerStarted","Data":"ccddb87fac52d2aed8c39a9106352b3f691aadac78060c565360687ffd645ef6"}
Sep 18 18:08:03.382758 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:03.382720    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq" event={"ID":"de81cfc0-ae09-4d04-8c06-0213ccc6d3c3","Type":"ContainerStarted","Data":"182feee01591c81967e8d49a23c80868ecb932fd64f7268cc9b6e16c1bc9ab5b"}
Sep 18 18:08:03.383183 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:03.382870    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:08:03.403281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:03.403229    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq" podStartSLOduration=2.068439912 podStartE2EDuration="9.403216395s" totalImagesPullingTime="7.334776483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:03.401194746 +0000 UTC m=+466.828407499" watchObservedRunningTime="2026-09-18 18:08:03.403216395 +0000 UTC m=+466.830429149"
Sep 18 18:08:09.416647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:09.416615    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-svhjq"
Sep 18 18:08:16.948593 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.948559    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-zzxwf"]
Sep 18 18:08:16.969454 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.969413    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-zzxwf"]
Sep 18 18:08:16.969633 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.969525    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:16.973439 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.973416    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 18:08:16.973854 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.973837    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 18:08:16.973938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.973921    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xxjjv\""
Sep 18 18:08:16.974687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:16.974668    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:08:17.030242 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.030210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.030389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.030248    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-metrics-certs\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.030389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.030293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqlpj\" (UniqueName: \"kubernetes.io/projected/89d9411b-20f5-4d24-8ac8-66a6392e1e32-kube-api-access-zqlpj\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.131645 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.131613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqlpj\" (UniqueName: \"kubernetes.io/projected/89d9411b-20f5-4d24-8ac8-66a6392e1e32-kube-api-access-zqlpj\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.131795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.131699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.131795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.131725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-metrics-certs\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.131874 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:08:17.131837    2582 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 18 18:08:17.131920 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:08:17.131908    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert podName:89d9411b-20f5-4d24-8ac8-66a6392e1e32 nodeName:}" failed. No retries permitted until 2026-09-18 18:08:17.631887054 +0000 UTC m=+481.059099789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert") pod "maas-controller-75dc6cbdf-zzxwf" (UID: "89d9411b-20f5-4d24-8ac8-66a6392e1e32") : secret "maas-controller-webhook-cert" not found
Sep 18 18:08:17.134158 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.134136    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-metrics-certs\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.147103 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.147083    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqlpj\" (UniqueName: \"kubernetes.io/projected/89d9411b-20f5-4d24-8ac8-66a6392e1e32-kube-api-access-zqlpj\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.636619 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.636584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.639085 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.639042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert\") pod \"maas-controller-75dc6cbdf-zzxwf\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") " pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:17.882917 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:17.882884    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:18.158149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:18.158120    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-zzxwf"]
Sep 18 18:08:18.160061 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:08:18.160021    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d9411b_20f5_4d24_8ac8_66a6392e1e32.slice/crio-dc9feeb99dfc7f8aa7fa985fe0b7485f6e2f11af6b29a631660a18e257d27399 WatchSource:0}: Error finding container dc9feeb99dfc7f8aa7fa985fe0b7485f6e2f11af6b29a631660a18e257d27399: Status 404 returned error can't find the container with id dc9feeb99dfc7f8aa7fa985fe0b7485f6e2f11af6b29a631660a18e257d27399
Sep 18 18:08:18.466735 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:18.466650    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf" event={"ID":"89d9411b-20f5-4d24-8ac8-66a6392e1e32","Type":"ContainerStarted","Data":"dc9feeb99dfc7f8aa7fa985fe0b7485f6e2f11af6b29a631660a18e257d27399"}
Sep 18 18:08:20.458251 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:20.458228    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:08:21.486881 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:21.486843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf" event={"ID":"89d9411b-20f5-4d24-8ac8-66a6392e1e32","Type":"ContainerStarted","Data":"7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf"}
Sep 18 18:08:21.487359 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:21.486951    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:21.526688 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:21.526645    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf" podStartSLOduration=3.232686139 podStartE2EDuration="5.52663269s" totalImagesPullingTime="2.293946551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:21.52306991 +0000 UTC m=+484.950282653" watchObservedRunningTime="2026-09-18 18:08:21.52663269 +0000 UTC m=+484.953845443"
Sep 18 18:08:31.926905 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:31.926871    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-zzxwf"]
Sep 18 18:08:31.927342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:31.927089    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf" podUID="89d9411b-20f5-4d24-8ac8-66a6392e1e32" containerName="manager" containerID="cri-o://7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf" gracePeriod=30
Sep 18 18:08:31.929162 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:31.929137    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:32.174093 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.174023    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:32.368797 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.368765    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert\") pod \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") "
Sep 18 18:08:32.368986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.368838    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqlpj\" (UniqueName: \"kubernetes.io/projected/89d9411b-20f5-4d24-8ac8-66a6392e1e32-kube-api-access-zqlpj\") pod \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") "
Sep 18 18:08:32.368986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.368915    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-metrics-certs\") pod \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\" (UID: \"89d9411b-20f5-4d24-8ac8-66a6392e1e32\") "
Sep 18 18:08:32.371186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.371156    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "89d9411b-20f5-4d24-8ac8-66a6392e1e32" (UID: "89d9411b-20f5-4d24-8ac8-66a6392e1e32"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:08:32.371316 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.371201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d9411b-20f5-4d24-8ac8-66a6392e1e32-kube-api-access-zqlpj" (OuterVolumeSpecName: "kube-api-access-zqlpj") pod "89d9411b-20f5-4d24-8ac8-66a6392e1e32" (UID: "89d9411b-20f5-4d24-8ac8-66a6392e1e32"). InnerVolumeSpecName "kube-api-access-zqlpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:08:32.371316 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.371224    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "89d9411b-20f5-4d24-8ac8-66a6392e1e32" (UID: "89d9411b-20f5-4d24-8ac8-66a6392e1e32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:08:32.470267 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.470241    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-webhook-cert\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:08:32.470267 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.470265    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqlpj\" (UniqueName: \"kubernetes.io/projected/89d9411b-20f5-4d24-8ac8-66a6392e1e32-kube-api-access-zqlpj\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:08:32.470267 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.470273    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d9411b-20f5-4d24-8ac8-66a6392e1e32-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:08:32.547458 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.547428    2582 generic.go:356] "Generic (PLEG): container finished" podID="89d9411b-20f5-4d24-8ac8-66a6392e1e32" containerID="7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf" exitCode=0
Sep 18 18:08:32.547588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.547478    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf"
Sep 18 18:08:32.547588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.547512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf" event={"ID":"89d9411b-20f5-4d24-8ac8-66a6392e1e32","Type":"ContainerDied","Data":"7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf"}
Sep 18 18:08:32.547588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.547567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-zzxwf" event={"ID":"89d9411b-20f5-4d24-8ac8-66a6392e1e32","Type":"ContainerDied","Data":"dc9feeb99dfc7f8aa7fa985fe0b7485f6e2f11af6b29a631660a18e257d27399"}
Sep 18 18:08:32.547588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.547588    2582 scope.go:117] "RemoveContainer" containerID="7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf"
Sep 18 18:08:32.557369 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.557352    2582 scope.go:117] "RemoveContainer" containerID="7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf"
Sep 18 18:08:32.557645 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:08:32.557630    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf\": container with ID starting with 7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf not found: ID does not exist" containerID="7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf"
Sep 18 18:08:32.557684 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.557653    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf"} err="failed to get container status \"7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf\": rpc error: code = NotFound desc = could not find container \"7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf\": container with ID starting with 7b5ce892b8ebd9df781d1846de11a99e2176c9af2188b67b16bacc2852f1dddf not found: ID does not exist"
Sep 18 18:08:32.569816 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.569789    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-zzxwf"]
Sep 18 18:08:32.574942 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.574920    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-zzxwf"]
Sep 18 18:08:32.919838 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:32.919809    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89d9411b-20f5-4d24-8ac8-66a6392e1e32" path="/var/lib/kubelet/pods/89d9411b-20f5-4d24-8ac8-66a6392e1e32/volumes"
Sep 18 18:08:55.989790 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:55.989754    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"]
Sep 18 18:08:55.990242 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:55.990193    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d9411b-20f5-4d24-8ac8-66a6392e1e32" containerName="manager"
Sep 18 18:08:55.990242 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:55.990208    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d9411b-20f5-4d24-8ac8-66a6392e1e32" containerName="manager"
Sep 18 18:08:55.990318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:55.990273    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d9411b-20f5-4d24-8ac8-66a6392e1e32" containerName="manager"
Sep 18 18:08:55.997886 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:55.997865    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.001012 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.000985    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 18:08:56.001720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.001701    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-s4cdg\""
Sep 18 18:08:56.007136 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.007104    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"]
Sep 18 18:08:56.013559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.013536    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"]
Sep 18 18:08:56.017163 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.017147    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.040722 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.040699    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"]
Sep 18 18:08:56.085812 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.085781    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/304380bf-7ce0-4153-9820-289d1483054f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gsz8b\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.085959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.085815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n52d\" (UniqueName: \"kubernetes.io/projected/304380bf-7ce0-4153-9820-289d1483054f-kube-api-access-8n52d\") pod \"payload-pre-processing-59fbc47bf8-gsz8b\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.085959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.085853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfzfk\" (UniqueName: \"kubernetes.io/projected/c832d106-d49c-4b1e-87b7-a0708eba82ca-kube-api-access-tfzfk\") pod \"payload-processing-5b54b7fbb7-tp5mm\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.085959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.085873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c832d106-d49c-4b1e-87b7-a0708eba82ca-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-tp5mm\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.187486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.187446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfzfk\" (UniqueName: \"kubernetes.io/projected/c832d106-d49c-4b1e-87b7-a0708eba82ca-kube-api-access-tfzfk\") pod \"payload-processing-5b54b7fbb7-tp5mm\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.187656 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.187503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c832d106-d49c-4b1e-87b7-a0708eba82ca-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-tp5mm\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.187656 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.187615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/304380bf-7ce0-4153-9820-289d1483054f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gsz8b\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.187656 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.187637    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n52d\" (UniqueName: \"kubernetes.io/projected/304380bf-7ce0-4153-9820-289d1483054f-kube-api-access-8n52d\") pod \"payload-pre-processing-59fbc47bf8-gsz8b\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.188249 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.188221    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c832d106-d49c-4b1e-87b7-a0708eba82ca-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-tp5mm\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.188323 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.188306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/304380bf-7ce0-4153-9820-289d1483054f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gsz8b\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.202293 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.202265    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfzfk\" (UniqueName: \"kubernetes.io/projected/c832d106-d49c-4b1e-87b7-a0708eba82ca-kube-api-access-tfzfk\") pod \"payload-processing-5b54b7fbb7-tp5mm\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.202979 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.202950    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n52d\" (UniqueName: \"kubernetes.io/projected/304380bf-7ce0-4153-9820-289d1483054f-kube-api-access-8n52d\") pod \"payload-pre-processing-59fbc47bf8-gsz8b\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.309635 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.309600    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:56.328136 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.328109    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:56.590017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.589983    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"]
Sep 18 18:08:56.591571 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:08:56.591540    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc832d106_d49c_4b1e_87b7_a0708eba82ca.slice/crio-cfad695ef00791cddda78af4450b926354a60c77cecb7b4a93b621f0879bcea2 WatchSource:0}: Error finding container cfad695ef00791cddda78af4450b926354a60c77cecb7b4a93b621f0879bcea2: Status 404 returned error can't find the container with id cfad695ef00791cddda78af4450b926354a60c77cecb7b4a93b621f0879bcea2
Sep 18 18:08:56.619563 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.619539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"]
Sep 18 18:08:56.621291 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:08:56.621268    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod304380bf_7ce0_4153_9820_289d1483054f.slice/crio-a6f955b981fa79dc789c4649061c124f0aeb656089070c1f1e95fffd6f7151fd WatchSource:0}: Error finding container a6f955b981fa79dc789c4649061c124f0aeb656089070c1f1e95fffd6f7151fd: Status 404 returned error can't find the container with id a6f955b981fa79dc789c4649061c124f0aeb656089070c1f1e95fffd6f7151fd
Sep 18 18:08:56.678530 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.678495    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm" event={"ID":"c832d106-d49c-4b1e-87b7-a0708eba82ca","Type":"ContainerStarted","Data":"cfad695ef00791cddda78af4450b926354a60c77cecb7b4a93b621f0879bcea2"}
Sep 18 18:08:56.679778 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:56.679747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b" event={"ID":"304380bf-7ce0-4153-9820-289d1483054f","Type":"ContainerStarted","Data":"a6f955b981fa79dc789c4649061c124f0aeb656089070c1f1e95fffd6f7151fd"}
Sep 18 18:08:58.695763 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:58.695614    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm" event={"ID":"c832d106-d49c-4b1e-87b7-a0708eba82ca","Type":"ContainerStarted","Data":"1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d"}
Sep 18 18:08:58.695763 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:58.695684    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:08:58.697370 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:58.697348    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b" event={"ID":"304380bf-7ce0-4153-9820-289d1483054f","Type":"ContainerStarted","Data":"d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170"}
Sep 18 18:08:58.697473 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:58.697391    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:08:58.713921 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:58.713879    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm" podStartSLOduration=2.11925016 podStartE2EDuration="3.713864519s" totalImagesPullingTime="1.594614359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:58.712789529 +0000 UTC m=+522.140002282" watchObservedRunningTime="2026-09-18 18:08:58.713864519 +0000 UTC m=+522.141077279"
Sep 18 18:08:58.729920 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:08:58.729864    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b" podStartSLOduration=2.140802303 podStartE2EDuration="3.729847566s" totalImagesPullingTime="1.589045263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:58.728842951 +0000 UTC m=+522.156055706" watchObservedRunningTime="2026-09-18 18:08:58.729847566 +0000 UTC m=+522.157060328"
Sep 18 18:09:04.248251 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:04.248204    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tvsjk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:09:04.248649 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:04.248303    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podUID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:09:05.746588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:05.746479    2582 generic.go:356] "Generic (PLEG): container finished" podID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerID="6f0c54371012eb2d3517266ab92eb9ed8c4072a599ea62af9a20714cbb9e058d" exitCode=0
Sep 18 18:09:05.746588 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:05.746551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" event={"ID":"7cc00339-98d4-4edd-8e78-8ebbae73a641","Type":"ContainerDied","Data":"6f0c54371012eb2d3517266ab92eb9ed8c4072a599ea62af9a20714cbb9e058d"}
Sep 18 18:09:05.747157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:05.746951    2582 scope.go:117] "RemoveContainer" containerID="6f0c54371012eb2d3517266ab92eb9ed8c4072a599ea62af9a20714cbb9e058d"
Sep 18 18:09:05.747974 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:05.747948    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:09:05.748095 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:05.748073    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:09:05.748157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:05.748129    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:09:06.759160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:06.759122    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" event={"ID":"7cc00339-98d4-4edd-8e78-8ebbae73a641","Type":"ContainerStarted","Data":"1cfa9b9f0dcdb0a56d07ddcc35be080c6a1132a15a2ccd6f19bada4904a7bddb"}
Sep 18 18:09:07.300733 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:07.300697    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:09:07.300913 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:07.300878    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tvsjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:09:07.300959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:07.300923    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podUID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 18 18:09:08.301776 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.301742    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tvsjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:09:08.302176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.301800    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podUID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 18 18:09:08.695492 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.695408    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"]
Sep 18 18:09:08.699729 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.699707    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.703439 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.703411    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v6bqs\""
Sep 18 18:09:08.703576 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.703467    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:09:08.703576 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.703414    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:09:08.703576 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.703420    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 18 18:09:08.710658 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.710636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"]
Sep 18 18:09:08.809103 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.809071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.809103 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.809102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d85144-59b8-41d2-b076-d689e04530e9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.809329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.809178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.809329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.809193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.809329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.809213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqhzc\" (UniqueName: \"kubernetes.io/projected/e4d85144-59b8-41d2-b076-d689e04530e9-kube-api-access-xqhzc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.809329 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.809302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.910638 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.910603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.910638 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.910637    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d85144-59b8-41d2-b076-d689e04530e9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.910862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.910688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.910862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.910703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.910862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.910723    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqhzc\" (UniqueName: \"kubernetes.io/projected/e4d85144-59b8-41d2-b076-d689e04530e9-kube-api-access-xqhzc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.910862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.910769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.911095 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.911028    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.911166 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.911137    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.911219 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.911174    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.913243 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.913220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d85144-59b8-41d2-b076-d689e04530e9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.913496 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.913478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d85144-59b8-41d2-b076-d689e04530e9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:08.920725 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:08.920704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqhzc\" (UniqueName: \"kubernetes.io/projected/e4d85144-59b8-41d2-b076-d689e04530e9-kube-api-access-xqhzc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx\" (UID: \"e4d85144-59b8-41d2-b076-d689e04530e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:09.011320 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.011215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:09.301548 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.301513    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tvsjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:09:09.301714 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.301570    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podUID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 18 18:09:09.309795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.309773    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"]
Sep 18 18:09:09.311528 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:09:09.311499    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d85144_59b8_41d2_b076_d689e04530e9.slice/crio-82c993f3dfa4a87a513d105b000611e6cdc528628fdff77399a06eb378db7ff3 WatchSource:0}: Error finding container 82c993f3dfa4a87a513d105b000611e6cdc528628fdff77399a06eb378db7ff3: Status 404 returned error can't find the container with id 82c993f3dfa4a87a513d105b000611e6cdc528628fdff77399a06eb378db7ff3
Sep 18 18:09:09.705348 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.705268    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:09:09.705348 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.705346    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:09:09.780077 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:09.779927    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx" event={"ID":"e4d85144-59b8-41d2-b076-d689e04530e9","Type":"ContainerStarted","Data":"82c993f3dfa4a87a513d105b000611e6cdc528628fdff77399a06eb378db7ff3"}
Sep 18 18:09:10.300865 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:10.300826    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tvsjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:09:10.301042 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:10.300897    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podUID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 18 18:09:11.301070 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:11.301020    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tvsjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 18 18:09:11.301462 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:11.301104    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk" podUID="7cc00339-98d4-4edd-8e78-8ebbae73a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 18 18:09:11.793127 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:11.793087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx" event={"ID":"e4d85144-59b8-41d2-b076-d689e04530e9","Type":"ContainerStarted","Data":"5fdd60b3c37c51998c9691393226078031b46119419d9a8fe14b94d0dece4267"}
Sep 18 18:09:11.814739 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:11.814692    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx" podStartSLOduration=2.40480985 podStartE2EDuration="3.814677853s" totalImagesPullingTime="1.409868003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:09:11.813684897 +0000 UTC m=+535.240897661" watchObservedRunningTime="2026-09-18 18:09:11.814677853 +0000 UTC m=+535.241890616"
Sep 18 18:09:12.304665 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:12.304637    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:09:12.305071 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:12.305002    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:09:12.305649 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:12.305631    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tvsjk"
Sep 18 18:09:19.011622 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:19.011590    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:19.012120 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:19.011784    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:19.024536 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:19.024515    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:19.849960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:19.849932    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jnsx"
Sep 18 18:09:55.683240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.683202    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"]
Sep 18 18:09:55.689015 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.688991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.692039 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.692013    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 18 18:09:55.699267 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.699243    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"]
Sep 18 18:09:55.855600 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.855567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.855789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.855609    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.855789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.855627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.855789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.855733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hfwr\" (UniqueName: \"kubernetes.io/projected/f1989961-df1d-4070-82a0-f79d7153be43-kube-api-access-8hfwr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.855789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.855784    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.855931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.855808    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1989961-df1d-4070-82a0-f79d7153be43-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956407 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hfwr\" (UniqueName: \"kubernetes.io/projected/f1989961-df1d-4070-82a0-f79d7153be43-kube-api-access-8hfwr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956407 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956407 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1989961-df1d-4070-82a0-f79d7153be43-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956892 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956864    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.956959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.956923    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.958951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.958929    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1989961-df1d-4070-82a0-f79d7153be43-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.959276 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.959257    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1989961-df1d-4070-82a0-f79d7153be43-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:55.967176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:55.967153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hfwr\" (UniqueName: \"kubernetes.io/projected/f1989961-df1d-4070-82a0-f79d7153be43-kube-api-access-8hfwr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn\" (UID: \"f1989961-df1d-4070-82a0-f79d7153be43\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:56.002045 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:56.002005    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:09:56.280642 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:56.280611    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"]
Sep 18 18:09:56.281678 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:09:56.281654    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1989961_df1d_4070_82a0_f79d7153be43.slice/crio-39f1b00591ef8e0f211c94ba3b08417db79161c87df072afdc6212dfe3a1d1f9 WatchSource:0}: Error finding container 39f1b00591ef8e0f211c94ba3b08417db79161c87df072afdc6212dfe3a1d1f9: Status 404 returned error can't find the container with id 39f1b00591ef8e0f211c94ba3b08417db79161c87df072afdc6212dfe3a1d1f9
Sep 18 18:09:57.057510 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:57.057476    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn" event={"ID":"f1989961-df1d-4070-82a0-f79d7153be43","Type":"ContainerStarted","Data":"1b6a9870c757f68ac194da91aeb0638a9cbcf94135fed2ac0de79c5fb86e7a81"}
Sep 18 18:09:57.057964 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:57.057530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn" event={"ID":"f1989961-df1d-4070-82a0-f79d7153be43","Type":"ContainerStarted","Data":"39f1b00591ef8e0f211c94ba3b08417db79161c87df072afdc6212dfe3a1d1f9"}
Sep 18 18:09:57.077041 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:09:57.076975    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn" podStartSLOduration=1.88340626 podStartE2EDuration="2.076954335s" totalImagesPullingTime="193.548075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:09:57.075272933 +0000 UTC m=+580.502485687" watchObservedRunningTime="2026-09-18 18:09:57.076954335 +0000 UTC m=+580.504167091"
Sep 18 18:10:06.003133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:06.003081    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:10:06.003609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:06.003162    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:10:06.015864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:06.015833    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:10:06.120367 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:06.120343    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pglcn"
Sep 18 18:10:18.824976 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:18.824945    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:10:18.826877 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:18.826853    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:10:43.029106 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.029043    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"]
Sep 18 18:10:43.032844 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.032824    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.035491 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.035469    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 18:10:43.046326 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.046299    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"]
Sep 18 18:10:43.178674 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.178635    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.178855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.178700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8337e0e0-391d-4cab-b449-6831e1d5a496-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.178855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.178782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnkcq\" (UniqueName: \"kubernetes.io/projected/8337e0e0-391d-4cab-b449-6831e1d5a496-kube-api-access-fnkcq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.178932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.178871    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.178932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.178901    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.178997 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.178937    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.279750 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.279722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.279906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.279767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8337e0e0-391d-4cab-b449-6831e1d5a496-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.279906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.279805    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnkcq\" (UniqueName: \"kubernetes.io/projected/8337e0e0-391d-4cab-b449-6831e1d5a496-kube-api-access-fnkcq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.279906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.279833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.279906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.279851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.279906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.279876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.280240 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.280210    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.280310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.280281    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.280373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.280339    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.282423 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.282402    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8337e0e0-391d-4cab-b449-6831e1d5a496-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.282610 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.282594    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8337e0e0-391d-4cab-b449-6831e1d5a496-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.289499 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.289472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnkcq\" (UniqueName: \"kubernetes.io/projected/8337e0e0-391d-4cab-b449-6831e1d5a496-kube-api-access-fnkcq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqjlg\" (UID: \"8337e0e0-391d-4cab-b449-6831e1d5a496\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.344610 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.344573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:43.628877 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.628775    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"]
Sep 18 18:10:43.636429 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:43.636405    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:10:44.336387 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:44.336351    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg" event={"ID":"8337e0e0-391d-4cab-b449-6831e1d5a496","Type":"ContainerStarted","Data":"85f70d2b025a0f1110e9e6ca1abe241e4ed76f2b7932da876ba6d8a4a28aa563"}
Sep 18 18:10:44.336795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:44.336416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg" event={"ID":"8337e0e0-391d-4cab-b449-6831e1d5a496","Type":"ContainerStarted","Data":"903833ca6dd1561343158ef13c3b3013fad134df2c7590786ff40db6b206622f"}
Sep 18 18:10:44.356429 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:44.356379    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg" podStartSLOduration=1.175982597 podStartE2EDuration="1.356364856s" totalImagesPullingTime="180.382259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:10:44.353452168 +0000 UTC m=+627.780664923" watchObservedRunningTime="2026-09-18 18:10:44.356364856 +0000 UTC m=+627.783577611"
Sep 18 18:10:53.345476 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:53.345430    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:53.345939 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:53.345506    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:53.358498 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:53.358471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:10:53.403171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:10:53.403142    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqjlg"
Sep 18 18:12:59.701861 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:12:59.701820    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"]
Sep 18 18:12:59.702703 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:12:59.702119    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm" podUID="c832d106-d49c-4b1e-87b7-a0708eba82ca" containerName="payload-processing" containerID="cri-o://1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d" gracePeriod=30
Sep 18 18:12:59.706607 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:12:59.706580    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"]
Sep 18 18:12:59.706999 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:12:59.706946    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b" podUID="304380bf-7ce0-4153-9820-289d1483054f" containerName="payload-pre-processing" containerID="cri-o://d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170" gracePeriod=30
Sep 18 18:12:59.989380 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:12:59.989300    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:12:59.992264 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:12:59.992243    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:13:00.120792 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.120756    2582 generic.go:356] "Generic (PLEG): container finished" podID="c832d106-d49c-4b1e-87b7-a0708eba82ca" containerID="1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d" exitCode=0
Sep 18 18:13:00.120960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.120802    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"
Sep 18 18:13:00.120960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.120843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm" event={"ID":"c832d106-d49c-4b1e-87b7-a0708eba82ca","Type":"ContainerDied","Data":"1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d"}
Sep 18 18:13:00.120960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.120895    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-tp5mm" event={"ID":"c832d106-d49c-4b1e-87b7-a0708eba82ca","Type":"ContainerDied","Data":"cfad695ef00791cddda78af4450b926354a60c77cecb7b4a93b621f0879bcea2"}
Sep 18 18:13:00.120960 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.120927    2582 scope.go:117] "RemoveContainer" containerID="1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d"
Sep 18 18:13:00.122154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.122137    2582 generic.go:356] "Generic (PLEG): container finished" podID="304380bf-7ce0-4153-9820-289d1483054f" containerID="d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170" exitCode=0
Sep 18 18:13:00.122239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.122165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b" event={"ID":"304380bf-7ce0-4153-9820-289d1483054f","Type":"ContainerDied","Data":"d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170"}
Sep 18 18:13:00.122239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.122182    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"
Sep 18 18:13:00.122239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.122196    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b" event={"ID":"304380bf-7ce0-4153-9820-289d1483054f","Type":"ContainerDied","Data":"a6f955b981fa79dc789c4649061c124f0aeb656089070c1f1e95fffd6f7151fd"}
Sep 18 18:13:00.130003 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.129984    2582 scope.go:117] "RemoveContainer" containerID="1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d"
Sep 18 18:13:00.130284 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:13:00.130266    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d\": container with ID starting with 1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d not found: ID does not exist" containerID="1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d"
Sep 18 18:13:00.130343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.130293    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d"} err="failed to get container status \"1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d\": rpc error: code = NotFound desc = could not find container \"1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d\": container with ID starting with 1c6ed24377078bcc023634509029b88907a6d6f23feb5fca8b76435c5f8fa55d not found: ID does not exist"
Sep 18 18:13:00.130343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.130310    2582 scope.go:117] "RemoveContainer" containerID="d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170"
Sep 18 18:13:00.133465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.133448    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/304380bf-7ce0-4153-9820-289d1483054f-plugins-config-volume\") pod \"304380bf-7ce0-4153-9820-289d1483054f\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") "
Sep 18 18:13:00.133542 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.133491    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c832d106-d49c-4b1e-87b7-a0708eba82ca-plugins-config-volume\") pod \"c832d106-d49c-4b1e-87b7-a0708eba82ca\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") "
Sep 18 18:13:00.133584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.133565    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfzfk\" (UniqueName: \"kubernetes.io/projected/c832d106-d49c-4b1e-87b7-a0708eba82ca-kube-api-access-tfzfk\") pod \"c832d106-d49c-4b1e-87b7-a0708eba82ca\" (UID: \"c832d106-d49c-4b1e-87b7-a0708eba82ca\") "
Sep 18 18:13:00.133628 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.133623    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n52d\" (UniqueName: \"kubernetes.io/projected/304380bf-7ce0-4153-9820-289d1483054f-kube-api-access-8n52d\") pod \"304380bf-7ce0-4153-9820-289d1483054f\" (UID: \"304380bf-7ce0-4153-9820-289d1483054f\") "
Sep 18 18:13:00.133846 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.133822    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/304380bf-7ce0-4153-9820-289d1483054f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "304380bf-7ce0-4153-9820-289d1483054f" (UID: "304380bf-7ce0-4153-9820-289d1483054f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:13:00.133922 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.133854    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c832d106-d49c-4b1e-87b7-a0708eba82ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c832d106-d49c-4b1e-87b7-a0708eba82ca" (UID: "c832d106-d49c-4b1e-87b7-a0708eba82ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:13:00.136037 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.136013    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/304380bf-7ce0-4153-9820-289d1483054f-kube-api-access-8n52d" (OuterVolumeSpecName: "kube-api-access-8n52d") pod "304380bf-7ce0-4153-9820-289d1483054f" (UID: "304380bf-7ce0-4153-9820-289d1483054f"). InnerVolumeSpecName "kube-api-access-8n52d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:00.136141 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.136042    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c832d106-d49c-4b1e-87b7-a0708eba82ca-kube-api-access-tfzfk" (OuterVolumeSpecName: "kube-api-access-tfzfk") pod "c832d106-d49c-4b1e-87b7-a0708eba82ca" (UID: "c832d106-d49c-4b1e-87b7-a0708eba82ca"). InnerVolumeSpecName "kube-api-access-tfzfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:00.139255 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.139235    2582 scope.go:117] "RemoveContainer" containerID="d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170"
Sep 18 18:13:00.139520 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:13:00.139502    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170\": container with ID starting with d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170 not found: ID does not exist" containerID="d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170"
Sep 18 18:13:00.139569 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.139528    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170"} err="failed to get container status \"d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170\": rpc error: code = NotFound desc = could not find container \"d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170\": container with ID starting with d38c98bdf508cf543398e6aac536e881780d396a480fa5cbc61a9905f6a56170 not found: ID does not exist"
Sep 18 18:13:00.234723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.234638    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n52d\" (UniqueName: \"kubernetes.io/projected/304380bf-7ce0-4153-9820-289d1483054f-kube-api-access-8n52d\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:13:00.234723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.234669    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/304380bf-7ce0-4153-9820-289d1483054f-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:13:00.234723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.234680    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c832d106-d49c-4b1e-87b7-a0708eba82ca-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:13:00.234723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.234688    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfzfk\" (UniqueName: \"kubernetes.io/projected/c832d106-d49c-4b1e-87b7-a0708eba82ca-kube-api-access-tfzfk\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:13:00.450650 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.450516    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"]
Sep 18 18:13:00.453209 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.453185    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gsz8b"]
Sep 18 18:13:00.467815 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.467790    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"]
Sep 18 18:13:00.471699 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.471679    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tp5mm"]
Sep 18 18:13:00.920490 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.920456    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="304380bf-7ce0-4153-9820-289d1483054f" path="/var/lib/kubelet/pods/304380bf-7ce0-4153-9820-289d1483054f/volumes"
Sep 18 18:13:00.920877 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.920862    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c832d106-d49c-4b1e-87b7-a0708eba82ca" path="/var/lib/kubelet/pods/c832d106-d49c-4b1e-87b7-a0708eba82ca/volumes"
Sep 18 18:13:00.949275 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949249    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dc7cc4dff-q27pw"]
Sep 18 18:13:00.949628 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949616    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="304380bf-7ce0-4153-9820-289d1483054f" containerName="payload-pre-processing"
Sep 18 18:13:00.949669 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949630    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="304380bf-7ce0-4153-9820-289d1483054f" containerName="payload-pre-processing"
Sep 18 18:13:00.949669 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949648    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c832d106-d49c-4b1e-87b7-a0708eba82ca" containerName="payload-processing"
Sep 18 18:13:00.949669 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949654    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c832d106-d49c-4b1e-87b7-a0708eba82ca" containerName="payload-processing"
Sep 18 18:13:00.949765 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949716    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c832d106-d49c-4b1e-87b7-a0708eba82ca" containerName="payload-processing"
Sep 18 18:13:00.949765 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.949727    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="304380bf-7ce0-4153-9820-289d1483054f" containerName="payload-pre-processing"
Sep 18 18:13:00.954216 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.954194    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:00.962916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.962895    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 18:13:00.963083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.962920    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 18:13:00.963083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.962927    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 18:13:00.963083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.962937    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xxjjv\""
Sep 18 18:13:00.979350 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:00.979290    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc7cc4dff-q27pw"]
Sep 18 18:13:01.041828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.041794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-metrics-certs\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.041981 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.041866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-webhook-cert\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.042157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.042135    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ltwt\" (UniqueName: \"kubernetes.io/projected/809f1117-84f1-498d-87fd-a163500439cd-kube-api-access-7ltwt\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.142903 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.142878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ltwt\" (UniqueName: \"kubernetes.io/projected/809f1117-84f1-498d-87fd-a163500439cd-kube-api-access-7ltwt\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.143096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.142912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-metrics-certs\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.143096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.142941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-webhook-cert\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.145473 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.145444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-webhook-cert\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.145614 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.145594    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-metrics-certs\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.155425 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.155405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ltwt\" (UniqueName: \"kubernetes.io/projected/809f1117-84f1-498d-87fd-a163500439cd-kube-api-access-7ltwt\") pod \"maas-controller-7dc7cc4dff-q27pw\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") " pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.265373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.265288    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:01.555179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:01.555155    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc7cc4dff-q27pw"]
Sep 18 18:13:01.556660 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:13:01.556638    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809f1117_84f1_498d_87fd_a163500439cd.slice/crio-6400fe65fc5850c88fe67db9baff7fdd6c2ad1c4f88bfc3d7574a7cbea512e71 WatchSource:0}: Error finding container 6400fe65fc5850c88fe67db9baff7fdd6c2ad1c4f88bfc3d7574a7cbea512e71: Status 404 returned error can't find the container with id 6400fe65fc5850c88fe67db9baff7fdd6c2ad1c4f88bfc3d7574a7cbea512e71
Sep 18 18:13:02.137955 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:02.137925    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw" event={"ID":"809f1117-84f1-498d-87fd-a163500439cd","Type":"ContainerStarted","Data":"6400fe65fc5850c88fe67db9baff7fdd6c2ad1c4f88bfc3d7574a7cbea512e71"}
Sep 18 18:13:03.145129 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:03.145087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw" event={"ID":"809f1117-84f1-498d-87fd-a163500439cd","Type":"ContainerStarted","Data":"6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df"}
Sep 18 18:13:03.145508 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:03.145198    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:03.170828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:03.170783    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw" podStartSLOduration=2.592950831 podStartE2EDuration="3.170769512s" totalImagesPullingTime="577.818681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:13:03.168280899 +0000 UTC m=+766.595493654" watchObservedRunningTime="2026-09-18 18:13:03.170769512 +0000 UTC m=+766.597982265"
Sep 18 18:13:14.153706 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.153674    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:13:14.513111 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.512997    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"]
Sep 18 18:13:14.516633 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.516613    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:14.520097 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.520071    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-gtxjw\""
Sep 18 18:13:14.520518 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.520501    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Sep 18 18:13:14.532836 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.532813    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"]
Sep 18 18:13:14.666779 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.666734    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwkh\" (UniqueName: \"kubernetes.io/projected/2f38b66e-5d7c-4fe8-96a9-b161806c53c1-kube-api-access-rhwkh\") pod \"ai-gateway-controller-79bdb6ccdd-24wqf\" (UID: \"2f38b66e-5d7c-4fe8-96a9-b161806c53c1\") " pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:14.768304 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.768215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwkh\" (UniqueName: \"kubernetes.io/projected/2f38b66e-5d7c-4fe8-96a9-b161806c53c1-kube-api-access-rhwkh\") pod \"ai-gateway-controller-79bdb6ccdd-24wqf\" (UID: \"2f38b66e-5d7c-4fe8-96a9-b161806c53c1\") " pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:14.782003 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.781970    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwkh\" (UniqueName: \"kubernetes.io/projected/2f38b66e-5d7c-4fe8-96a9-b161806c53c1-kube-api-access-rhwkh\") pod \"ai-gateway-controller-79bdb6ccdd-24wqf\" (UID: \"2f38b66e-5d7c-4fe8-96a9-b161806c53c1\") " pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:14.827165 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:14.827134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:15.108345 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:15.108315    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"]
Sep 18 18:13:15.108958 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:13:15.108924    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f38b66e_5d7c_4fe8_96a9_b161806c53c1.slice/crio-f73a865b22ae685521ca4524967f2fd1bd623d2a910e87e2a3015b4964488170 WatchSource:0}: Error finding container f73a865b22ae685521ca4524967f2fd1bd623d2a910e87e2a3015b4964488170: Status 404 returned error can't find the container with id f73a865b22ae685521ca4524967f2fd1bd623d2a910e87e2a3015b4964488170
Sep 18 18:13:15.214619 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:15.214589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf" event={"ID":"2f38b66e-5d7c-4fe8-96a9-b161806c53c1","Type":"ContainerStarted","Data":"f73a865b22ae685521ca4524967f2fd1bd623d2a910e87e2a3015b4964488170"}
Sep 18 18:13:17.229264 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:17.229164    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf" event={"ID":"2f38b66e-5d7c-4fe8-96a9-b161806c53c1","Type":"ContainerStarted","Data":"2d6a5abbc332e88382c3e4657b4af4c599ebb04ff3772551eee9fc3111868394"}
Sep 18 18:13:17.229594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:17.229258    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:17.266458 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:17.266405    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf" podStartSLOduration=1.5873426940000002 podStartE2EDuration="3.266389748s" totalImagesPullingTime="1.679047054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:13:17.264245282 +0000 UTC m=+780.691458037" watchObservedRunningTime="2026-09-18 18:13:17.266389748 +0000 UTC m=+780.693602569"
Sep 18 18:13:21.469217 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.469182    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"]
Sep 18 18:13:21.474450 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.474426    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.477043 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.477013    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 18:13:21.477226 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.477141    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cndrx\""
Sep 18 18:13:21.489296 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.489267    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"]
Sep 18 18:13:21.499362 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.499334    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-85fd65c94c-ttfd5"]
Sep 18 18:13:21.502991 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.502958    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.516836 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.516811    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-85fd65c94c-ttfd5"]
Sep 18 18:13:21.531362 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.531333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhkmq\" (UniqueName: \"kubernetes.io/projected/4277a873-c982-4563-9f2c-306b90371db0-kube-api-access-bhkmq\") pod \"payload-processing-85fd65c94c-ttfd5\" (UID: \"4277a873-c982-4563-9f2c-306b90371db0\") " pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.531480 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.531403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1150832c-dd30-44ad-9a79-1bf56352abf0-plugins-config-volume\") pod \"payload-pre-processing-76f657cdc4-59w9r\" (UID: \"1150832c-dd30-44ad-9a79-1bf56352abf0\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.531565 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.531500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4277a873-c982-4563-9f2c-306b90371db0-plugins-config-volume\") pod \"payload-processing-85fd65c94c-ttfd5\" (UID: \"4277a873-c982-4563-9f2c-306b90371db0\") " pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.531565 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.531558    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqkm7\" (UniqueName: \"kubernetes.io/projected/1150832c-dd30-44ad-9a79-1bf56352abf0-kube-api-access-qqkm7\") pod \"payload-pre-processing-76f657cdc4-59w9r\" (UID: \"1150832c-dd30-44ad-9a79-1bf56352abf0\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.632474 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.632430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhkmq\" (UniqueName: \"kubernetes.io/projected/4277a873-c982-4563-9f2c-306b90371db0-kube-api-access-bhkmq\") pod \"payload-processing-85fd65c94c-ttfd5\" (UID: \"4277a873-c982-4563-9f2c-306b90371db0\") " pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.632655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.632495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1150832c-dd30-44ad-9a79-1bf56352abf0-plugins-config-volume\") pod \"payload-pre-processing-76f657cdc4-59w9r\" (UID: \"1150832c-dd30-44ad-9a79-1bf56352abf0\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.632655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.632554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4277a873-c982-4563-9f2c-306b90371db0-plugins-config-volume\") pod \"payload-processing-85fd65c94c-ttfd5\" (UID: \"4277a873-c982-4563-9f2c-306b90371db0\") " pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.632655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.632600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqkm7\" (UniqueName: \"kubernetes.io/projected/1150832c-dd30-44ad-9a79-1bf56352abf0-kube-api-access-qqkm7\") pod \"payload-pre-processing-76f657cdc4-59w9r\" (UID: \"1150832c-dd30-44ad-9a79-1bf56352abf0\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.633395 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.633375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1150832c-dd30-44ad-9a79-1bf56352abf0-plugins-config-volume\") pod \"payload-pre-processing-76f657cdc4-59w9r\" (UID: \"1150832c-dd30-44ad-9a79-1bf56352abf0\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.633444 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.633434    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4277a873-c982-4563-9f2c-306b90371db0-plugins-config-volume\") pod \"payload-processing-85fd65c94c-ttfd5\" (UID: \"4277a873-c982-4563-9f2c-306b90371db0\") " pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.652907 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.652886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhkmq\" (UniqueName: \"kubernetes.io/projected/4277a873-c982-4563-9f2c-306b90371db0-kube-api-access-bhkmq\") pod \"payload-processing-85fd65c94c-ttfd5\" (UID: \"4277a873-c982-4563-9f2c-306b90371db0\") " pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:21.653079 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.653037    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqkm7\" (UniqueName: \"kubernetes.io/projected/1150832c-dd30-44ad-9a79-1bf56352abf0-kube-api-access-qqkm7\") pod \"payload-pre-processing-76f657cdc4-59w9r\" (UID: \"1150832c-dd30-44ad-9a79-1bf56352abf0\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.784912 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.784884    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:21.813961 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:21.813933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:22.066309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:22.066282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"]
Sep 18 18:13:22.067342 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:13:22.067298    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1150832c_dd30_44ad_9a79_1bf56352abf0.slice/crio-eae66036e5ef276ac8cec102849f4c10f2d7eaf10151eef78cc1b22d5ad4ae3d WatchSource:0}: Error finding container eae66036e5ef276ac8cec102849f4c10f2d7eaf10151eef78cc1b22d5ad4ae3d: Status 404 returned error can't find the container with id eae66036e5ef276ac8cec102849f4c10f2d7eaf10151eef78cc1b22d5ad4ae3d
Sep 18 18:13:22.100862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:22.100838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-85fd65c94c-ttfd5"]
Sep 18 18:13:22.101727 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:13:22.101704    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4277a873_c982_4563_9f2c_306b90371db0.slice/crio-ae43ace01856348d1c51967ba5ea0d0d1e8653a38efe6b6e51bee4e7a4d6a1c3 WatchSource:0}: Error finding container ae43ace01856348d1c51967ba5ea0d0d1e8653a38efe6b6e51bee4e7a4d6a1c3: Status 404 returned error can't find the container with id ae43ace01856348d1c51967ba5ea0d0d1e8653a38efe6b6e51bee4e7a4d6a1c3
Sep 18 18:13:22.260730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:22.260690    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r" event={"ID":"1150832c-dd30-44ad-9a79-1bf56352abf0","Type":"ContainerStarted","Data":"eae66036e5ef276ac8cec102849f4c10f2d7eaf10151eef78cc1b22d5ad4ae3d"}
Sep 18 18:13:22.261902 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:22.261870    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5" event={"ID":"4277a873-c982-4563-9f2c-306b90371db0","Type":"ContainerStarted","Data":"ae43ace01856348d1c51967ba5ea0d0d1e8653a38efe6b6e51bee4e7a4d6a1c3"}
Sep 18 18:13:24.280446 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:24.280406    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r" event={"ID":"1150832c-dd30-44ad-9a79-1bf56352abf0","Type":"ContainerStarted","Data":"2e019320d69b0d986ad85acc5d93066ef652e48d833281bcc3938795dec379ef"}
Sep 18 18:13:24.280848 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:24.280471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:24.282068 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:24.282016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5" event={"ID":"4277a873-c982-4563-9f2c-306b90371db0","Type":"ContainerStarted","Data":"8e7ad2dd1053b3034668b813ef50b3695f0d2c2976513323f7fbf9a7bf0cfcc7"}
Sep 18 18:13:24.282181 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:24.282101    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:24.306926 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:24.306874    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r" podStartSLOduration=1.9436594029999998 podStartE2EDuration="3.306858858s" totalImagesPullingTime="1.363199455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:13:24.304321661 +0000 UTC m=+787.731534414" watchObservedRunningTime="2026-09-18 18:13:24.306858858 +0000 UTC m=+787.734071611"
Sep 18 18:13:24.333167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:24.333129    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5" podStartSLOduration=2.006393659 podStartE2EDuration="3.333115184s" totalImagesPullingTime="1.326721525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:13:24.331369482 +0000 UTC m=+787.758582236" watchObservedRunningTime="2026-09-18 18:13:24.333115184 +0000 UTC m=+787.760327955"
Sep 18 18:13:28.237851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:28.237820    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:35.290454 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:35.290416    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-76f657cdc4-59w9r"
Sep 18 18:13:35.290954 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:35.290499    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-85fd65c94c-ttfd5"
Sep 18 18:13:42.267086 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.267040    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"]
Sep 18 18:13:42.267483 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.267241    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf" podUID="2f38b66e-5d7c-4fe8-96a9-b161806c53c1" containerName="manager" containerID="cri-o://2d6a5abbc332e88382c3e4657b4af4c599ebb04ff3772551eee9fc3111868394" gracePeriod=30
Sep 18 18:13:42.387522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.387484    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f38b66e-5d7c-4fe8-96a9-b161806c53c1" containerID="2d6a5abbc332e88382c3e4657b4af4c599ebb04ff3772551eee9fc3111868394" exitCode=0
Sep 18 18:13:42.387641 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.387559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf" event={"ID":"2f38b66e-5d7c-4fe8-96a9-b161806c53c1","Type":"ContainerDied","Data":"2d6a5abbc332e88382c3e4657b4af4c599ebb04ff3772551eee9fc3111868394"}
Sep 18 18:13:42.513429 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.513409    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:42.625299 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.625267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhwkh\" (UniqueName: \"kubernetes.io/projected/2f38b66e-5d7c-4fe8-96a9-b161806c53c1-kube-api-access-rhwkh\") pod \"2f38b66e-5d7c-4fe8-96a9-b161806c53c1\" (UID: \"2f38b66e-5d7c-4fe8-96a9-b161806c53c1\") "
Sep 18 18:13:42.627827 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.627797    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f38b66e-5d7c-4fe8-96a9-b161806c53c1-kube-api-access-rhwkh" (OuterVolumeSpecName: "kube-api-access-rhwkh") pod "2f38b66e-5d7c-4fe8-96a9-b161806c53c1" (UID: "2f38b66e-5d7c-4fe8-96a9-b161806c53c1"). InnerVolumeSpecName "kube-api-access-rhwkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:42.726205 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:42.726179    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhwkh\" (UniqueName: \"kubernetes.io/projected/2f38b66e-5d7c-4fe8-96a9-b161806c53c1-kube-api-access-rhwkh\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:13:43.394709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:43.394674    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf" event={"ID":"2f38b66e-5d7c-4fe8-96a9-b161806c53c1","Type":"ContainerDied","Data":"f73a865b22ae685521ca4524967f2fd1bd623d2a910e87e2a3015b4964488170"}
Sep 18 18:13:43.394709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:43.394700    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"
Sep 18 18:13:43.395262 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:43.394749    2582 scope.go:117] "RemoveContainer" containerID="2d6a5abbc332e88382c3e4657b4af4c599ebb04ff3772551eee9fc3111868394"
Sep 18 18:13:43.412257 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:43.412230    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"]
Sep 18 18:13:43.418802 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:43.418779    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-79bdb6ccdd-24wqf"]
Sep 18 18:13:44.920640 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:13:44.920609    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f38b66e-5d7c-4fe8-96a9-b161806c53c1" path="/var/lib/kubelet/pods/2f38b66e-5d7c-4fe8-96a9-b161806c53c1/volumes"
Sep 18 18:14:02.046665 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.046631    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"]
Sep 18 18:14:02.047177 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.047161    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f38b66e-5d7c-4fe8-96a9-b161806c53c1" containerName="manager"
Sep 18 18:14:02.047177 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.047179    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f38b66e-5d7c-4fe8-96a9-b161806c53c1" containerName="manager"
Sep 18 18:14:02.047293 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.047283    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f38b66e-5d7c-4fe8-96a9-b161806c53c1" containerName="manager"
Sep 18 18:14:02.049407 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.049390    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.052627 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.052607    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-37c153-openshift-default-dockercfg-krs7p\""
Sep 18 18:14:02.065378 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.065354    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"]
Sep 18 18:14:02.097041 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-envoy\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097268 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097091    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-credential-socket\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097268 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097232    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-token\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097409 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097266    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-podinfo\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097470 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-socket\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-data\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097579 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097521    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istiod-ca-cert\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097579 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-certs\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.097724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.097594    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcvf6\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-kube-api-access-jcvf6\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.198653 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-token\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.198653 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-podinfo\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.198903 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-socket\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.198903 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-data\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.198903 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istiod-ca-cert\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199099 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-certs\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199099 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcvf6\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-kube-api-access-jcvf6\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199099 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.198991    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-envoy\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199099 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.199069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-credential-socket\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199332 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.199142    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-socket\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199332 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.199179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-data\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199453 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.199384    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-credential-socket\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199662 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.199639    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-certs\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.199729 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.199644    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istiod-ca-cert\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.201464 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.201441    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-envoy\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.201599 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.201529    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-podinfo\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.208904 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.208874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-token\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.209345 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.209326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcvf6\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-kube-api-access-jcvf6\") pod \"e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") " pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.360918 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.360834    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:02.680461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.680421    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"]
Sep 18 18:14:02.681785 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:02.681763    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f32b99_d00c_4f0b_a6a8_35d71b659bd4.slice/crio-8ac043449074b92df96adf8b9e50676830bd200d507e8a04407a69e5eaaf6b1a WatchSource:0}: Error finding container 8ac043449074b92df96adf8b9e50676830bd200d507e8a04407a69e5eaaf6b1a: Status 404 returned error can't find the container with id 8ac043449074b92df96adf8b9e50676830bd200d507e8a04407a69e5eaaf6b1a
Sep 18 18:14:02.684002 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.683973    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:02.684094 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.684040    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:02.684094 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:02.684086    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:03.510256 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:03.510223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" event={"ID":"48f32b99-d00c-4f0b-a6a8-35d71b659bd4","Type":"ContainerStarted","Data":"b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"}
Sep 18 18:14:03.510745 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:03.510272    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" event={"ID":"48f32b99-d00c-4f0b-a6a8-35d71b659bd4","Type":"ContainerStarted","Data":"8ac043449074b92df96adf8b9e50676830bd200d507e8a04407a69e5eaaf6b1a"}
Sep 18 18:14:03.532712 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:03.532664    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podStartSLOduration=1.532649065 podStartE2EDuration="1.532649065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:03.531006371 +0000 UTC m=+826.958219139" watchObservedRunningTime="2026-09-18 18:14:03.532649065 +0000 UTC m=+826.959861819"
Sep 18 18:14:04.361704 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:04.361672    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:04.366952 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:04.366927    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:04.516760 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:04.516726    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:04.517724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:04.517702    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:07.688607 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.688574    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"]
Sep 18 18:14:07.691043 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.691028    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.693836 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.693816    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-37c153-dockercfg-v6vwk\""
Sep 18 18:14:07.694144 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.694125    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-37c153\""
Sep 18 18:14:07.707369 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.707342    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"]
Sep 18 18:14:07.735937 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.735909    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"]
Sep 18 18:14:07.738584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.738563    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:07.753155 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.753130    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8aee3bb0-5d28-45f3-a374-c507854901f3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:07.753273 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.753195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/8aee3bb0-5d28-45f3-a374-c507854901f3-kube-api-access-n57fd\") pod \"payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:07.753273 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.753242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0efd2b7a-6791-471b-818e-5de06768e5da-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.753434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.753410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg6vn\" (UniqueName: \"kubernetes.io/projected/0efd2b7a-6791-471b-818e-5de06768e5da-kube-api-access-wg6vn\") pod \"payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.755956 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.755935    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"]
Sep 18 18:14:07.854408 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.854373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg6vn\" (UniqueName: \"kubernetes.io/projected/0efd2b7a-6791-471b-818e-5de06768e5da-kube-api-access-wg6vn\") pod \"payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.854601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.854418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8aee3bb0-5d28-45f3-a374-c507854901f3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:07.854601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.854448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/8aee3bb0-5d28-45f3-a374-c507854901f3-kube-api-access-n57fd\") pod \"payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:07.854601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.854483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0efd2b7a-6791-471b-818e-5de06768e5da-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.855186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.855162    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8aee3bb0-5d28-45f3-a374-c507854901f3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:07.855268 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.855187    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0efd2b7a-6791-471b-818e-5de06768e5da-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.863709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.863679    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg6vn\" (UniqueName: \"kubernetes.io/projected/0efd2b7a-6791-471b-818e-5de06768e5da-kube-api-access-wg6vn\") pod \"payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:07.863841 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:07.863722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/8aee3bb0-5d28-45f3-a374-c507854901f3-kube-api-access-n57fd\") pod \"payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:08.003271 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.003180    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:08.048794 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.048763    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:08.285693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.285666    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"]
Sep 18 18:14:08.286345 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:08.286318    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efd2b7a_6791_471b_818e_5de06768e5da.slice/crio-90cf6b190c6ac7fb3e3f586c23b1c34c3e5f6a18bf1d5276738c6b0438035944 WatchSource:0}: Error finding container 90cf6b190c6ac7fb3e3f586c23b1c34c3e5f6a18bf1d5276738c6b0438035944: Status 404 returned error can't find the container with id 90cf6b190c6ac7fb3e3f586c23b1c34c3e5f6a18bf1d5276738c6b0438035944
Sep 18 18:14:08.357997 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.357966    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"]
Sep 18 18:14:08.361200 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:08.361146    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aee3bb0_5d28_45f3_a374_c507854901f3.slice/crio-880a44353876173611269ae3ab18a9379644ab6718727151d2d8b74da1541ebe WatchSource:0}: Error finding container 880a44353876173611269ae3ab18a9379644ab6718727151d2d8b74da1541ebe: Status 404 returned error can't find the container with id 880a44353876173611269ae3ab18a9379644ab6718727151d2d8b74da1541ebe
Sep 18 18:14:08.556541 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.556439    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml" event={"ID":"8aee3bb0-5d28-45f3-a374-c507854901f3","Type":"ContainerStarted","Data":"f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670"}
Sep 18 18:14:08.556541 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.556513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml" event={"ID":"8aee3bb0-5d28-45f3-a374-c507854901f3","Type":"ContainerStarted","Data":"880a44353876173611269ae3ab18a9379644ab6718727151d2d8b74da1541ebe"}
Sep 18 18:14:08.556763 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.556686    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:08.558418 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.558394    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb" event={"ID":"0efd2b7a-6791-471b-818e-5de06768e5da","Type":"ContainerStarted","Data":"25529e1013580e5ac11c0745461120c222d450fc932410c7b795daf96cbba113"}
Sep 18 18:14:08.558545 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.558441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb" event={"ID":"0efd2b7a-6791-471b-818e-5de06768e5da","Type":"ContainerStarted","Data":"90cf6b190c6ac7fb3e3f586c23b1c34c3e5f6a18bf1d5276738c6b0438035944"}
Sep 18 18:14:08.558545 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.558471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:08.598312 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.598252    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml" podStartSLOduration=1.598229997 podStartE2EDuration="1.598229997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:08.597065923 +0000 UTC m=+832.024278674" watchObservedRunningTime="2026-09-18 18:14:08.598229997 +0000 UTC m=+832.025442761"
Sep 18 18:14:08.644517 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:08.644462    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb" podStartSLOduration=1.6444400620000001 podStartE2EDuration="1.644440062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:08.641906497 +0000 UTC m=+832.069119252" watchObservedRunningTime="2026-09-18 18:14:08.644440062 +0000 UTC m=+832.071652817"
Sep 18 18:14:11.991365 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:11.991327    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"]
Sep 18 18:14:11.994188 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:11.994169    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:11.996791 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:11.996765    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-de7535-dockercfg-zw6nt\""
Sep 18 18:14:11.996899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:11.996813    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-de7535\""
Sep 18 18:14:12.009333 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.009310    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"]
Sep 18 18:14:12.100946 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.100916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.101155 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.100969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq9kb\" (UniqueName: \"kubernetes.io/projected/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-kube-api-access-xq9kb\") pod \"payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.201575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.201539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.201741 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.201602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq9kb\" (UniqueName: \"kubernetes.io/projected/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-kube-api-access-xq9kb\") pod \"payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.202519 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.202492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.213021 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.212995    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq9kb\" (UniqueName: \"kubernetes.io/projected/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-kube-api-access-xq9kb\") pod \"payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.306371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.306338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:12.598251 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.598220    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"]
Sep 18 18:14:12.599084 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:12.599025    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b1dbfb_f6a9_43aa_b26a_c0cbfb40234c.slice/crio-af2953bec483b8d78e0183769ec9733404337fb88afb7eaa3f3691c82860129b WatchSource:0}: Error finding container af2953bec483b8d78e0183769ec9733404337fb88afb7eaa3f3691c82860129b: Status 404 returned error can't find the container with id af2953bec483b8d78e0183769ec9733404337fb88afb7eaa3f3691c82860129b
Sep 18 18:14:12.903504 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.903407    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"]
Sep 18 18:14:12.906377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.906358    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:12.909350 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.909332    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7fbf2b1b-openshift-default-dockercfg-nbtmr\""
Sep 18 18:14:12.923873 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:12.923840    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"]
Sep 18 18:14:13.009933 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.009892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-credential-socket\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.009954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-certs\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010015    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft58c\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-kube-api-access-ft58c\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010145    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-socket\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-podinfo\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-token\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010268    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-envoy\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010342    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fce592b-73f2-4230-a568-1fe86a803ce5-istiod-ca-cert\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.010400 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.010384    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-data\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111015 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.110980    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-socket\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-podinfo\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111083    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-token\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-envoy\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111201 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111185    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fce592b-73f2-4230-a568-1fe86a803ce5-istiod-ca-cert\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111411 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-data\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111411 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-credential-socket\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111411 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-certs\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111411 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft58c\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-kube-api-access-ft58c\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111679 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-socket\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.111757 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111660    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-data\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.112012 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111990    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fce592b-73f2-4230-a568-1fe86a803ce5-istiod-ca-cert\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.112115 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.111946    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-credential-socket\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.112345 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.112325    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-certs\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.113683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.113663    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-envoy\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.113853 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.113832    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-podinfo\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.120785 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.120761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft58c\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-kube-api-access-ft58c\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.121189 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.121176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-token\") pod \"e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") " pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.218469 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.218394    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:13.523381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.523345    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"]
Sep 18 18:14:13.525657 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:13.525629    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fce592b_73f2_4230_a568_1fe86a803ce5.slice/crio-8dfb1122542865cf7f723e71bc39f104c23621e128b859e0ada58fd54c253801 WatchSource:0}: Error finding container 8dfb1122542865cf7f723e71bc39f104c23621e128b859e0ada58fd54c253801: Status 404 returned error can't find the container with id 8dfb1122542865cf7f723e71bc39f104c23621e128b859e0ada58fd54c253801
Sep 18 18:14:13.534076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.528630    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:13.534076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.528787    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:13.534076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.528879    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:13.594520 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.594489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" event={"ID":"1fce592b-73f2-4230-a568-1fe86a803ce5","Type":"ContainerStarted","Data":"8dfb1122542865cf7f723e71bc39f104c23621e128b859e0ada58fd54c253801"}
Sep 18 18:14:13.596298 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.596272    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9" event={"ID":"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c","Type":"ContainerStarted","Data":"1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c"}
Sep 18 18:14:13.596416 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.596331    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9" event={"ID":"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c","Type":"ContainerStarted","Data":"af2953bec483b8d78e0183769ec9733404337fb88afb7eaa3f3691c82860129b"}
Sep 18 18:14:13.596416 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.596379    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:13.630619 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:13.630568    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9" podStartSLOduration=2.6305516620000002 podStartE2EDuration="2.630551662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:13.629154049 +0000 UTC m=+837.056366802" watchObservedRunningTime="2026-09-18 18:14:13.630551662 +0000 UTC m=+837.057764415"
Sep 18 18:14:14.604257 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:14.604223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" event={"ID":"1fce592b-73f2-4230-a568-1fe86a803ce5","Type":"ContainerStarted","Data":"57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"}
Sep 18 18:14:14.629655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:14.629585    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podStartSLOduration=2.6295623949999998 podStartE2EDuration="2.629562395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:14.624667164 +0000 UTC m=+838.051879918" watchObservedRunningTime="2026-09-18 18:14:14.629562395 +0000 UTC m=+838.056775149"
Sep 18 18:14:15.218693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:15.218660    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:15.223491 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:15.223457    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:15.610334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:15.610297    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:15.611350 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:15.611326    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:18.879911 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:18.879880    2582 scope.go:117] "RemoveContainer" containerID="62d1a98efe69b34ead2c251249a1aaeedf14070cd86ba96b259bf9d56ee25852"
Sep 18 18:14:19.568123 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:19.568090    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:14:19.568530 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:19.568463    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:14:22.236019 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.235979    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"]
Sep 18 18:14:22.239998 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.239969    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.242749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.242720    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7fbf2b1b-dockercfg-4xnzc\""
Sep 18 18:14:22.242749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.242740    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7fbf2b1b\""
Sep 18 18:14:22.256232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.256208    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"]
Sep 18 18:14:22.324740 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.324700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s9k9\" (UniqueName: \"kubernetes.io/projected/4a8500af-5865-476c-a843-ac499f17aaae-kube-api-access-4s9k9\") pod \"payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") " pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.324902 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.324771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8500af-5865-476c-a843-ac499f17aaae-plugins-config-volume\") pod \"payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") " pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.426285 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.426244    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8500af-5865-476c-a843-ac499f17aaae-plugins-config-volume\") pod \"payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") " pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.426477 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.426393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s9k9\" (UniqueName: \"kubernetes.io/projected/4a8500af-5865-476c-a843-ac499f17aaae-kube-api-access-4s9k9\") pod \"payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") " pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.427043 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.427010    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8500af-5865-476c-a843-ac499f17aaae-plugins-config-volume\") pod \"payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") " pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.438895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.438866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s9k9\" (UniqueName: \"kubernetes.io/projected/4a8500af-5865-476c-a843-ac499f17aaae-kube-api-access-4s9k9\") pod \"payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") " pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.555468 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.555439    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:22.854651 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:22.854610    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"]
Sep 18 18:14:22.855875 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:22.855835    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8500af_5865_476c_a843_ac499f17aaae.slice/crio-289bc6468ecec3d12e3a2e4443012441fa60b56b95e4cf3ac37058cdf0ea9445 WatchSource:0}: Error finding container 289bc6468ecec3d12e3a2e4443012441fa60b56b95e4cf3ac37058cdf0ea9445: Status 404 returned error can't find the container with id 289bc6468ecec3d12e3a2e4443012441fa60b56b95e4cf3ac37058cdf0ea9445
Sep 18 18:14:23.695306 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.695216    2582 generic.go:356] "Generic (PLEG): container finished" podID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerID="b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c" exitCode=0
Sep 18 18:14:23.695766 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.695295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" event={"ID":"48f32b99-d00c-4f0b-a6a8-35d71b659bd4","Type":"ContainerDied","Data":"b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"}
Sep 18 18:14:23.695839 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.695800    2582 scope.go:117] "RemoveContainer" containerID="b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"
Sep 18 18:14:23.697132 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.697097    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:23.697245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.697185    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:23.697245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.697228    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:23.697319 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.697282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4" event={"ID":"4a8500af-5865-476c-a843-ac499f17aaae","Type":"ContainerStarted","Data":"9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78"}
Sep 18 18:14:23.697353 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.697328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4" event={"ID":"4a8500af-5865-476c-a843-ac499f17aaae","Type":"ContainerStarted","Data":"289bc6468ecec3d12e3a2e4443012441fa60b56b95e4cf3ac37058cdf0ea9445"}
Sep 18 18:14:23.697388 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.697353    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:23.781678 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:23.781619    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4" podStartSLOduration=1.781598459 podStartE2EDuration="1.781598459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:23.776889971 +0000 UTC m=+847.204102720" watchObservedRunningTime="2026-09-18 18:14:23.781598459 +0000 UTC m=+847.208811214"
Sep 18 18:14:24.605940 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:24.605914    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:14:24.707361 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:24.707329    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" event={"ID":"48f32b99-d00c-4f0b-a6a8-35d71b659bd4","Type":"ContainerStarted","Data":"901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6"}
Sep 18 18:14:25.361225 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:25.361189    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:25.361431 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:25.361390    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 18:14:25.361505 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:25.361445    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 18:14:26.362295 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:26.362257    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 18:14:26.362667 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:26.362320    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 18:14:27.361385 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:27.361345    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 18:14:27.361573 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:27.361405    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 18:14:28.361469 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:28.361433    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 18:14:28.361847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:28.361485    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 18:14:29.362001 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:29.361970    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 18 18:14:29.362399 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:29.362027    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 18 18:14:30.365272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:30.365243    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:30.365646 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:30.365523    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:30.366379 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:30.366359    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:14:31.869182 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.869146    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"]
Sep 18 18:14:31.872349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.872332    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:31.875026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.875002    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-bcf02f\""
Sep 18 18:14:31.875026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.875021    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-bcf02f-dockercfg-87rs5\""
Sep 18 18:14:31.885514 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.885490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"]
Sep 18 18:14:31.948075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.945631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:31.948075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:31.945743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdbk5\" (UniqueName: \"kubernetes.io/projected/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-kube-api-access-wdbk5\") pod \"payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.050076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.049651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.050076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.049775    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdbk5\" (UniqueName: \"kubernetes.io/projected/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-kube-api-access-wdbk5\") pod \"payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.054072 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.050911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.059513 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.059492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdbk5\" (UniqueName: \"kubernetes.io/projected/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-kube-api-access-wdbk5\") pod \"payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.183922 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.183837    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.474485 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.474457    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"]
Sep 18 18:14:32.475552 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:32.475521    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bac72a_3a9d_4586_9d27_5a87cfa5ac57.slice/crio-f052263b97c0ae2316836ac38641542d39c166607482668328285abbceb902a7 WatchSource:0}: Error finding container f052263b97c0ae2316836ac38641542d39c166607482668328285abbceb902a7: Status 404 returned error can't find the container with id f052263b97c0ae2316836ac38641542d39c166607482668328285abbceb902a7
Sep 18 18:14:32.775944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.775710    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424" event={"ID":"89bac72a-3a9d-4586-9d27-5a87cfa5ac57","Type":"ContainerStarted","Data":"0d9d0290a3dd8cdb4aaaaccdda88eaf67bcbae730865ba9b58217981d1bc069a"}
Sep 18 18:14:32.775944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.775757    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424" event={"ID":"89bac72a-3a9d-4586-9d27-5a87cfa5ac57","Type":"ContainerStarted","Data":"f052263b97c0ae2316836ac38641542d39c166607482668328285abbceb902a7"}
Sep 18 18:14:32.775944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.775833    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:32.791436 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:32.791387    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424" podStartSLOduration=1.791367833 podStartE2EDuration="1.791367833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:32.790509342 +0000 UTC m=+856.217722098" watchObservedRunningTime="2026-09-18 18:14:32.791367833 +0000 UTC m=+856.218580588"
Sep 18 18:14:34.709575 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:34.709544    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:40.830999 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:40.830965    2582 generic.go:356] "Generic (PLEG): container finished" podID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerID="57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2" exitCode=0
Sep 18 18:14:40.831419 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:40.831024    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" event={"ID":"1fce592b-73f2-4230-a568-1fe86a803ce5","Type":"ContainerDied","Data":"57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"}
Sep 18 18:14:40.831419 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:40.831388    2582 scope.go:117] "RemoveContainer" containerID="57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"
Sep 18 18:14:40.832466 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:40.832440    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:40.832532 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:40.832505    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:40.832574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:40.832541    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 18 18:14:41.847418 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:41.847370    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" event={"ID":"1fce592b-73f2-4230-a568-1fe86a803ce5","Type":"ContainerStarted","Data":"c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede"}
Sep 18 18:14:42.218902 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:42.218826    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:42.220227 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:42.220199    2582 patch_prober.go:28] interesting pod/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:14:42.220346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:42.220249    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 18 18:14:43.219747 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:43.219712    2582 patch_prober.go:28] interesting pod/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:14:43.220173 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:43.219770    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 18 18:14:43.785554 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:43.785519    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:14:44.218908 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:44.218822    2582 patch_prober.go:28] interesting pod/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:14:44.218908 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:44.218884    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 18 18:14:45.218971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:45.218938    2582 patch_prober.go:28] interesting pod/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:14:45.219370 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:45.218990    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 18 18:14:45.610822 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:45.610783    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:46.219091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.219030    2582 patch_prober.go:28] interesting pod/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:14:46.219465 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.219113    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 18 18:14:46.682822 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.682781    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"]
Sep 18 18:14:46.686858 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.686825    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:46.690112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.689738    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:14:46.694381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.694352    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"]
Sep 18 18:14:46.707922 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.707887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4261cc62-de6c-4b89-b1a9-bd0abb0e0565-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh\" (UID: \"4261cc62-de6c-4b89-b1a9-bd0abb0e0565\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:46.809163 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.809135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4261cc62-de6c-4b89-b1a9-bd0abb0e0565-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh\" (UID: \"4261cc62-de6c-4b89-b1a9-bd0abb0e0565\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:46.809730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:46.809709    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4261cc62-de6c-4b89-b1a9-bd0abb0e0565-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh\" (UID: \"4261cc62-de6c-4b89-b1a9-bd0abb0e0565\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:47.002309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:47.002231    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:47.223634 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:47.223593    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:47.224721 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:47.224699    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:14:47.283192 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:47.283158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"]
Sep 18 18:14:47.286396 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:14:47.286313    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4261cc62_de6c_4b89_b1a9_bd0abb0e0565.slice/crio-9b53febfaf7ab87ac62743e789f082a1a6e9c8bae16e11de8468e6030f263596 WatchSource:0}: Error finding container 9b53febfaf7ab87ac62743e789f082a1a6e9c8bae16e11de8468e6030f263596: Status 404 returned error can't find the container with id 9b53febfaf7ab87ac62743e789f082a1a6e9c8bae16e11de8468e6030f263596
Sep 18 18:14:47.887521 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:47.887483    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh" event={"ID":"4261cc62-de6c-4b89-b1a9-bd0abb0e0565","Type":"ContainerStarted","Data":"9b53febfaf7ab87ac62743e789f082a1a6e9c8bae16e11de8468e6030f263596"}
Sep 18 18:14:50.908172 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:50.908081    2582 generic.go:356] "Generic (PLEG): container finished" podID="4261cc62-de6c-4b89-b1a9-bd0abb0e0565" containerID="00de11bb3e062834c837845d728eccc0a2f438dc9cc6dda172ae9d3944dd6c53" exitCode=0
Sep 18 18:14:50.908172 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:50.908136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh" event={"ID":"4261cc62-de6c-4b89-b1a9-bd0abb0e0565","Type":"ContainerDied","Data":"00de11bb3e062834c837845d728eccc0a2f438dc9cc6dda172ae9d3944dd6c53"}
Sep 18 18:14:52.061219 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.061187    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:52.170740 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.170661    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4261cc62-de6c-4b89-b1a9-bd0abb0e0565-maas-api-service-ca\") pod \"4261cc62-de6c-4b89-b1a9-bd0abb0e0565\" (UID: \"4261cc62-de6c-4b89-b1a9-bd0abb0e0565\") "
Sep 18 18:14:52.170998 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.170977    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4261cc62-de6c-4b89-b1a9-bd0abb0e0565-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4261cc62-de6c-4b89-b1a9-bd0abb0e0565" (UID: "4261cc62-de6c-4b89-b1a9-bd0abb0e0565"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:14:52.271818 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.271779    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4261cc62-de6c-4b89-b1a9-bd0abb0e0565-maas-api-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:14:52.922264 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.922234    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"
Sep 18 18:14:52.922431 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.922265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh" event={"ID":"4261cc62-de6c-4b89-b1a9-bd0abb0e0565","Type":"ContainerDied","Data":"9b53febfaf7ab87ac62743e789f082a1a6e9c8bae16e11de8468e6030f263596"}
Sep 18 18:14:52.922431 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:52.922310    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b53febfaf7ab87ac62743e789f082a1a6e9c8bae16e11de8468e6030f263596"
Sep 18 18:14:58.101947 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.101901    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"]
Sep 18 18:14:58.102853 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.102807    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4" podUID="4a8500af-5865-476c-a843-ac499f17aaae" containerName="payload-processing" containerID="cri-o://9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78" gracePeriod=30
Sep 18 18:14:58.293072 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.293009    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"]
Sep 18 18:14:58.299623 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.299586    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7fbf2b1b-lgmqh"]
Sep 18 18:14:58.383432 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.383394    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:58.427255 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.427214    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"]
Sep 18 18:14:58.427974 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.427739    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" containerID="cri-o://c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede" gracePeriod=30
Sep 18 18:14:58.533089 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.533039    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s9k9\" (UniqueName: \"kubernetes.io/projected/4a8500af-5865-476c-a843-ac499f17aaae-kube-api-access-4s9k9\") pod \"4a8500af-5865-476c-a843-ac499f17aaae\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") "
Sep 18 18:14:58.533319 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.533299    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8500af-5865-476c-a843-ac499f17aaae-plugins-config-volume\") pod \"4a8500af-5865-476c-a843-ac499f17aaae\" (UID: \"4a8500af-5865-476c-a843-ac499f17aaae\") "
Sep 18 18:14:58.533728 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.533661    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a8500af-5865-476c-a843-ac499f17aaae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a8500af-5865-476c-a843-ac499f17aaae" (UID: "4a8500af-5865-476c-a843-ac499f17aaae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:14:58.533842 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.533824    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8500af-5865-476c-a843-ac499f17aaae-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.537921 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.537842    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a8500af-5865-476c-a843-ac499f17aaae-kube-api-access-4s9k9" (OuterVolumeSpecName: "kube-api-access-4s9k9") pod "4a8500af-5865-476c-a843-ac499f17aaae" (UID: "4a8500af-5865-476c-a843-ac499f17aaae"). InnerVolumeSpecName "kube-api-access-4s9k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:14:58.634616 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.634530    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s9k9\" (UniqueName: \"kubernetes.io/projected/4a8500af-5865-476c-a843-ac499f17aaae-kube-api-access-4s9k9\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.924690 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.924535    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4261cc62-de6c-4b89-b1a9-bd0abb0e0565" path="/var/lib/kubelet/pods/4261cc62-de6c-4b89-b1a9-bd0abb0e0565/volumes"
Sep 18 18:14:58.959864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.959831    2582 generic.go:356] "Generic (PLEG): container finished" podID="4a8500af-5865-476c-a843-ac499f17aaae" containerID="9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78" exitCode=0
Sep 18 18:14:58.960006 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.959880    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"
Sep 18 18:14:58.960006 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.959908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4" event={"ID":"4a8500af-5865-476c-a843-ac499f17aaae","Type":"ContainerDied","Data":"9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78"}
Sep 18 18:14:58.960006 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.959959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4" event={"ID":"4a8500af-5865-476c-a843-ac499f17aaae","Type":"ContainerDied","Data":"289bc6468ecec3d12e3a2e4443012441fa60b56b95e4cf3ac37058cdf0ea9445"}
Sep 18 18:14:58.960006 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.959980    2582 scope.go:117] "RemoveContainer" containerID="9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78"
Sep 18 18:14:58.969391 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.969373    2582 scope.go:117] "RemoveContainer" containerID="9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78"
Sep 18 18:14:58.969670 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:14:58.969650    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78\": container with ID starting with 9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78 not found: ID does not exist" containerID="9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78"
Sep 18 18:14:58.969713 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.969679    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78"} err="failed to get container status \"9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78\": rpc error: code = NotFound desc = could not find container \"9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78\": container with ID starting with 9952174e18801d2d3250e80cba63a951dcc398c917f9e797462b56d2ae920f78 not found: ID does not exist"
Sep 18 18:14:58.981628 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.981603    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"]
Sep 18 18:14:58.989389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:14:58.989370    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7fbf2b1b-58fb765575-z5vm4"]
Sep 18 18:15:00.205154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205121    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"]
Sep 18 18:15:00.205618 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205603    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a8500af-5865-476c-a843-ac499f17aaae" containerName="payload-processing"
Sep 18 18:15:00.205618 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205620    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a8500af-5865-476c-a843-ac499f17aaae" containerName="payload-processing"
Sep 18 18:15:00.205705 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205632    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4261cc62-de6c-4b89-b1a9-bd0abb0e0565" containerName="revoke-keys"
Sep 18 18:15:00.205705 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205638    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4261cc62-de6c-4b89-b1a9-bd0abb0e0565" containerName="revoke-keys"
Sep 18 18:15:00.205769 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205715    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4261cc62-de6c-4b89-b1a9-bd0abb0e0565" containerName="revoke-keys"
Sep 18 18:15:00.205769 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.205727    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a8500af-5865-476c-a843-ac499f17aaae" containerName="payload-processing"
Sep 18 18:15:00.208733 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.208716    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"
Sep 18 18:15:00.212368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.212344    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"]
Sep 18 18:15:00.213849 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.213830    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:15:00.214752 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.214735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"
Sep 18 18:15:00.220579 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.220562    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"
Sep 18 18:15:00.227754 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.227732    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"
Sep 18 18:15:00.229605 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.229582    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"]
Sep 18 18:15:00.232832 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.232810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"
Sep 18 18:15:00.240269 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.239609    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"]
Sep 18 18:15:00.245936 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.245078    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"]
Sep 18 18:15:00.279265 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.279181    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"]
Sep 18 18:15:00.292761 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.292740    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"
Sep 18 18:15:00.539003 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.538972    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"]
Sep 18 18:15:00.540620 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:15:00.540596    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad645a5b_2893_4f21_b5b5_7f7f757fd00f.slice/crio-1ed06ddbdbd2ac67b7e2982e4aac4d32dabdc2f42e5987d7f18952de4ca2487f WatchSource:0}: Error finding container 1ed06ddbdbd2ac67b7e2982e4aac4d32dabdc2f42e5987d7f18952de4ca2487f: Status 404 returned error can't find the container with id 1ed06ddbdbd2ac67b7e2982e4aac4d32dabdc2f42e5987d7f18952de4ca2487f
Sep 18 18:15:00.542099 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:15:00.542072    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4e7c0d_2dc9_4741_ae55_9d5093cf8db1.slice/crio-7acb3877500886b33e9f4bc58096a3661def8ebc9d2c9855e4380821f91ecbf9 WatchSource:0}: Error finding container 7acb3877500886b33e9f4bc58096a3661def8ebc9d2c9855e4380821f91ecbf9: Status 404 returned error can't find the container with id 7acb3877500886b33e9f4bc58096a3661def8ebc9d2c9855e4380821f91ecbf9
Sep 18 18:15:00.542345 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.542263    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"]
Sep 18 18:15:00.592906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.592874    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"]
Sep 18 18:15:00.594245 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:15:00.594214    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3f83e7_e3a1_4b82_825e_2a348aa7c801.slice/crio-610fcfa65f5547a72440379238b82f4c56c77ab439a2683f7ec22222708a2799 WatchSource:0}: Error finding container 610fcfa65f5547a72440379238b82f4c56c77ab439a2683f7ec22222708a2799: Status 404 returned error can't find the container with id 610fcfa65f5547a72440379238b82f4c56c77ab439a2683f7ec22222708a2799
Sep 18 18:15:00.925229 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.923546    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a8500af-5865-476c-a843-ac499f17aaae" path="/var/lib/kubelet/pods/4a8500af-5865-476c-a843-ac499f17aaae/volumes"
Sep 18 18:15:00.974830 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.974794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt" event={"ID":"fc3f83e7-e3a1-4b82-825e-2a348aa7c801","Type":"ContainerStarted","Data":"610fcfa65f5547a72440379238b82f4c56c77ab439a2683f7ec22222708a2799"}
Sep 18 18:15:00.976550 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.976522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq" event={"ID":"ad645a5b-2893-4f21-b5b5-7f7f757fd00f","Type":"ContainerStarted","Data":"b0212fac1eec968d17c55e7519b695970a60cc1424f0205f211ac7ef5a76d686"}
Sep 18 18:15:00.976655 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.976569    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq" event={"ID":"ad645a5b-2893-4f21-b5b5-7f7f757fd00f","Type":"ContainerStarted","Data":"1ed06ddbdbd2ac67b7e2982e4aac4d32dabdc2f42e5987d7f18952de4ca2487f"}
Sep 18 18:15:00.978320 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.978294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf" event={"ID":"8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1","Type":"ContainerStarted","Data":"167e7c036d69dcaa9b8ea8bdd07570157ceeef5123ca440429c40cf6b1191e58"}
Sep 18 18:15:00.978415 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.978337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf" event={"ID":"8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1","Type":"ContainerStarted","Data":"7acb3877500886b33e9f4bc58096a3661def8ebc9d2c9855e4380821f91ecbf9"}
Sep 18 18:15:00.994415 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:00.994371    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq" podStartSLOduration=0.994357513 podStartE2EDuration="994.357513ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:00.993709585 +0000 UTC m=+884.420922340" watchObservedRunningTime="2026-09-18 18:15:00.994357513 +0000 UTC m=+884.421570267"
Sep 18 18:15:01.009211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.009171    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf" podStartSLOduration=1.009157068 podStartE2EDuration="1.009157068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:01.007225428 +0000 UTC m=+884.434438181" watchObservedRunningTime="2026-09-18 18:15:01.009157068 +0000 UTC m=+884.436369840"
Sep 18 18:15:01.692110 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:15:01.692075    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3f83e7_e3a1_4b82_825e_2a348aa7c801.slice/crio-conmon-7e80632c4a64df192ca53d0ff05717e9ad1d412b4ff0e71a226e1e44b0d53b5a.scope\": RecentStats: unable to find data in memory cache]"
Sep 18 18:15:01.986865 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.986832    2582 generic.go:356] "Generic (PLEG): container finished" podID="fc3f83e7-e3a1-4b82-825e-2a348aa7c801" containerID="7e80632c4a64df192ca53d0ff05717e9ad1d412b4ff0e71a226e1e44b0d53b5a" exitCode=0
Sep 18 18:15:01.987090 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.986931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt" event={"ID":"fc3f83e7-e3a1-4b82-825e-2a348aa7c801","Type":"ContainerDied","Data":"7e80632c4a64df192ca53d0ff05717e9ad1d412b4ff0e71a226e1e44b0d53b5a"}
Sep 18 18:15:01.988544 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.988521    2582 generic.go:356] "Generic (PLEG): container finished" podID="ad645a5b-2893-4f21-b5b5-7f7f757fd00f" containerID="b0212fac1eec968d17c55e7519b695970a60cc1424f0205f211ac7ef5a76d686" exitCode=0
Sep 18 18:15:01.988681 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.988602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq" event={"ID":"ad645a5b-2893-4f21-b5b5-7f7f757fd00f","Type":"ContainerDied","Data":"b0212fac1eec968d17c55e7519b695970a60cc1424f0205f211ac7ef5a76d686"}
Sep 18 18:15:01.990175 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.990154    2582 generic.go:356] "Generic (PLEG): container finished" podID="8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1" containerID="167e7c036d69dcaa9b8ea8bdd07570157ceeef5123ca440429c40cf6b1191e58" exitCode=0
Sep 18 18:15:01.990262 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:01.990179    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf" event={"ID":"8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1","Type":"ContainerDied","Data":"167e7c036d69dcaa9b8ea8bdd07570157ceeef5123ca440429c40cf6b1191e58"}
Sep 18 18:15:02.226568 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:02.226530    2582 patch_prober.go:28] interesting pod/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:15:02.226779 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:02.226596    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:15:03.170486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.170443    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"
Sep 18 18:15:03.216024 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.215975    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"
Sep 18 18:15:03.220994 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.220973    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"
Sep 18 18:15:03.679349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.679322    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:15:03.795426 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795392    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft58c\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-kube-api-access-ft58c\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795432    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fce592b-73f2-4230-a568-1fe86a803ce5-istiod-ca-cert\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795467    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-data\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795511    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-podinfo\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795535    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-token\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795580    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-certs\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795623    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-credential-socket\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795665    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-socket\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795719    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-envoy\") pod \"1fce592b-73f2-4230-a568-1fe86a803ce5\" (UID: \"1fce592b-73f2-4230-a568-1fe86a803ce5\") "
Sep 18 18:15:03.795869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795813    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:03.795869 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.795858    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fce592b-73f2-4230-a568-1fe86a803ce5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:15:03.796149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796008    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:03.796149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796078    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:03.796149 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796118    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:03.796331 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796294    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fce592b-73f2-4230-a568-1fe86a803ce5-istiod-ca-cert\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.796331 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796320    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-data\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.796435 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796334    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.796435 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796344    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-credential-socket\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.796435 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.796358    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-workload-socket\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.797798 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.797777    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-kube-api-access-ft58c" (OuterVolumeSpecName: "kube-api-access-ft58c") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "kube-api-access-ft58c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:15:03.798096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.798078    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:03.798288 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.798267    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:15:03.798343 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.798271    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fce592b-73f2-4230-a568-1fe86a803ce5" (UID: "1fce592b-73f2-4230-a568-1fe86a803ce5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:15:03.896951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.896919    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft58c\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-kube-api-access-ft58c\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.896951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.896946    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-podinfo\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.896951 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.896955    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-token\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:03.897195 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:03.896965    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fce592b-73f2-4230-a568-1fe86a803ce5-istio-envoy\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:04.011537 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.011492    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"
Sep 18 18:15:04.011720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.011536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf" event={"ID":"8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1","Type":"ContainerDied","Data":"7acb3877500886b33e9f4bc58096a3661def8ebc9d2c9855e4380821f91ecbf9"}
Sep 18 18:15:04.011720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.011633    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7acb3877500886b33e9f4bc58096a3661def8ebc9d2c9855e4380821f91ecbf9"
Sep 18 18:15:04.013450 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.013420    2582 generic.go:356] "Generic (PLEG): container finished" podID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerID="c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede" exitCode=0
Sep 18 18:15:04.013594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.013476    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"
Sep 18 18:15:04.013594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.013501    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" event={"ID":"1fce592b-73f2-4230-a568-1fe86a803ce5","Type":"ContainerDied","Data":"c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede"}
Sep 18 18:15:04.013594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.013557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl" event={"ID":"1fce592b-73f2-4230-a568-1fe86a803ce5","Type":"ContainerDied","Data":"8dfb1122542865cf7f723e71bc39f104c23621e128b859e0ada58fd54c253801"}
Sep 18 18:15:04.013594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.013586    2582 scope.go:117] "RemoveContainer" containerID="c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede"
Sep 18 18:15:04.015148 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.015080    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt" event={"ID":"fc3f83e7-e3a1-4b82-825e-2a348aa7c801","Type":"ContainerDied","Data":"610fcfa65f5547a72440379238b82f4c56c77ab439a2683f7ec22222708a2799"}
Sep 18 18:15:04.015148 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.015125    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="610fcfa65f5547a72440379238b82f4c56c77ab439a2683f7ec22222708a2799"
Sep 18 18:15:04.015332 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.015241    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"
Sep 18 18:15:04.016702 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.016678    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq" event={"ID":"ad645a5b-2893-4f21-b5b5-7f7f757fd00f","Type":"ContainerDied","Data":"1ed06ddbdbd2ac67b7e2982e4aac4d32dabdc2f42e5987d7f18952de4ca2487f"}
Sep 18 18:15:04.016702 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.016699    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-p4cxq"
Sep 18 18:15:04.016892 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.016714    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ed06ddbdbd2ac67b7e2982e4aac4d32dabdc2f42e5987d7f18952de4ca2487f"
Sep 18 18:15:04.029610 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.029519    2582 scope.go:117] "RemoveContainer" containerID="57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"
Sep 18 18:15:04.039200 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.039181    2582 scope.go:117] "RemoveContainer" containerID="c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede"
Sep 18 18:15:04.039474 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:15:04.039450    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede\": container with ID starting with c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede not found: ID does not exist" containerID="c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede"
Sep 18 18:15:04.039547 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.039485    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede"} err="failed to get container status \"c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede\": rpc error: code = NotFound desc = could not find container \"c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede\": container with ID starting with c657e49c338b1dc28ef9080be9e50b8cd70adedc6f558fd1c194de6dfcb62ede not found: ID does not exist"
Sep 18 18:15:04.039547 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.039509    2582 scope.go:117] "RemoveContainer" containerID="57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"
Sep 18 18:15:04.039756 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:15:04.039738    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2\": container with ID starting with 57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2 not found: ID does not exist" containerID="57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"
Sep 18 18:15:04.039816 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.039760    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2"} err="failed to get container status \"57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2\": rpc error: code = NotFound desc = could not find container \"57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2\": container with ID starting with 57217c48a59d5a2b7ec37a233aa2a2b6b338a7fe51c3779a7ffed6aca11fcfa2 not found: ID does not exist"
Sep 18 18:15:04.047891 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.047837    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"]
Sep 18 18:15:04.050754 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.050732    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7fbf2b1b-openshift-default-5579d78456-hdtbl"]
Sep 18 18:15:04.920959 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:04.920922    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" path="/var/lib/kubelet/pods/1fce592b-73f2-4230-a568-1fe86a803ce5/volumes"
Sep 18 18:15:18.878793 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:18.878765    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:15:18.879830 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:18.879812    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:15:27.178620 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.178590    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"]
Sep 18 18:15:27.179083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179069    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy"
Sep 18 18:15:27.179083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179085    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy"
Sep 18 18:15:27.179167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179100    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad645a5b-2893-4f21-b5b5-7f7f757fd00f" containerName="cleanup"
Sep 18 18:15:27.179167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179107    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad645a5b-2893-4f21-b5b5-7f7f757fd00f" containerName="cleanup"
Sep 18 18:15:27.179167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179124    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3f83e7-e3a1-4b82-825e-2a348aa7c801" containerName="cleanup"
Sep 18 18:15:27.179167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179129    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3f83e7-e3a1-4b82-825e-2a348aa7c801" containerName="cleanup"
Sep 18 18:15:27.179167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179138    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1" containerName="cleanup"
Sep 18 18:15:27.179167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179143    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1" containerName="cleanup"
Sep 18 18:15:27.179362 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179221    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy"
Sep 18 18:15:27.179362 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179230    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1" containerName="cleanup"
Sep 18 18:15:27.179362 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179236    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3f83e7-e3a1-4b82-825e-2a348aa7c801" containerName="cleanup"
Sep 18 18:15:27.179362 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.179244    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad645a5b-2893-4f21-b5b5-7f7f757fd00f" containerName="cleanup"
Sep 18 18:15:27.188388 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.188359    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:27.188601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.188543    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"]
Sep 18 18:15:27.190907 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.190881    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:15:27.303236 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.303193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c527b-0ad1-403f-80b8-12473b2ba53e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2e816045-bb477\" (UID: \"841c527b-0ad1-403f-80b8-12473b2ba53e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:27.404257 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.404215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c527b-0ad1-403f-80b8-12473b2ba53e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2e816045-bb477\" (UID: \"841c527b-0ad1-403f-80b8-12473b2ba53e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:27.404894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.404871    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c527b-0ad1-403f-80b8-12473b2ba53e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2e816045-bb477\" (UID: \"841c527b-0ad1-403f-80b8-12473b2ba53e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:27.501632 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.501556    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:27.776907 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:27.776839    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"]
Sep 18 18:15:27.779896 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:15:27.779856    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841c527b_0ad1_403f_80b8_12473b2ba53e.slice/crio-4a745c77f81d8b92c8a82feff140a8da2ddd997ca730e87a5108ef599a4214db WatchSource:0}: Error finding container 4a745c77f81d8b92c8a82feff140a8da2ddd997ca730e87a5108ef599a4214db: Status 404 returned error can't find the container with id 4a745c77f81d8b92c8a82feff140a8da2ddd997ca730e87a5108ef599a4214db
Sep 18 18:15:28.177002 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:28.176966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477" event={"ID":"841c527b-0ad1-403f-80b8-12473b2ba53e","Type":"ContainerStarted","Data":"2527339ad97d7f5b49bc039762abd1c3a4b70d7b29cb7d56e5aa03c6ac55d6a5"}
Sep 18 18:15:28.177176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:28.177074    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477" event={"ID":"841c527b-0ad1-403f-80b8-12473b2ba53e","Type":"ContainerStarted","Data":"4a745c77f81d8b92c8a82feff140a8da2ddd997ca730e87a5108ef599a4214db"}
Sep 18 18:15:28.194975 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:28.194930    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477" podStartSLOduration=1.194915967 podStartE2EDuration="1.194915967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:28.193991765 +0000 UTC m=+911.621204520" watchObservedRunningTime="2026-09-18 18:15:28.194915967 +0000 UTC m=+911.622128720"
Sep 18 18:15:29.185592 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:29.185556    2582 generic.go:356] "Generic (PLEG): container finished" podID="841c527b-0ad1-403f-80b8-12473b2ba53e" containerID="2527339ad97d7f5b49bc039762abd1c3a4b70d7b29cb7d56e5aa03c6ac55d6a5" exitCode=0
Sep 18 18:15:29.185762 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:29.185646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477" event={"ID":"841c527b-0ad1-403f-80b8-12473b2ba53e","Type":"ContainerDied","Data":"2527339ad97d7f5b49bc039762abd1c3a4b70d7b29cb7d56e5aa03c6ac55d6a5"}
Sep 18 18:15:30.336165 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:30.336141    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:30.431806 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:30.431775    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c527b-0ad1-403f-80b8-12473b2ba53e-maas-api-service-ca\") pod \"841c527b-0ad1-403f-80b8-12473b2ba53e\" (UID: \"841c527b-0ad1-403f-80b8-12473b2ba53e\") "
Sep 18 18:15:30.432150 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:30.432124    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/841c527b-0ad1-403f-80b8-12473b2ba53e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "841c527b-0ad1-403f-80b8-12473b2ba53e" (UID: "841c527b-0ad1-403f-80b8-12473b2ba53e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:15:30.533158 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:30.533128    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841c527b-0ad1-403f-80b8-12473b2ba53e-maas-api-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:15:31.200272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:31.200180    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477" event={"ID":"841c527b-0ad1-403f-80b8-12473b2ba53e","Type":"ContainerDied","Data":"4a745c77f81d8b92c8a82feff140a8da2ddd997ca730e87a5108ef599a4214db"}
Sep 18 18:15:31.200272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:31.200203    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"
Sep 18 18:15:31.200272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:31.200229    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a745c77f81d8b92c8a82feff140a8da2ddd997ca730e87a5108ef599a4214db"
Sep 18 18:15:34.108328 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:34.108274    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"]
Sep 18 18:15:34.111038 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:34.111015    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2e816045-bb477"]
Sep 18 18:15:34.920859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:34.920815    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="841c527b-0ad1-403f-80b8-12473b2ba53e" path="/var/lib/kubelet/pods/841c527b-0ad1-403f-80b8-12473b2ba53e/volumes"
Sep 18 18:15:38.391977 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.391939    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"]
Sep 18 18:15:38.392743 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.392720    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy"
Sep 18 18:15:38.392789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.392748    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy"
Sep 18 18:15:38.392789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.392773    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841c527b-0ad1-403f-80b8-12473b2ba53e" containerName="revoke-keys"
Sep 18 18:15:38.392789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.392781    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="841c527b-0ad1-403f-80b8-12473b2ba53e" containerName="revoke-keys"
Sep 18 18:15:38.392949 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.392937    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fce592b-73f2-4230-a568-1fe86a803ce5" containerName="istio-proxy"
Sep 18 18:15:38.392985 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.392957    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="841c527b-0ad1-403f-80b8-12473b2ba53e" containerName="revoke-keys"
Sep 18 18:15:38.396859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.396835    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.399310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.399280    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-89dea9e1-serving-cert\""
Sep 18 18:15:38.400257 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.400233    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:15:38.400373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.400308    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jkbfn\""
Sep 18 18:15:38.406699 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.406669    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"]
Sep 18 18:15:38.511597 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.511557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p556\" (UniqueName: \"kubernetes.io/projected/cf58f785-a232-4130-bcc8-09226178623a-kube-api-access-8p556\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.511779 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.511605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf58f785-a232-4130-bcc8-09226178623a-openshift-service-ca\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.511779 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.511649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.511877 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.511793    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-metrics-certs\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.613079 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.613018    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.613258 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.613148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-metrics-certs\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.613258 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:15:38.613171    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-serving-cert: secret "maas-api-e2e-ait-89dea9e1-serving-cert" not found
Sep 18 18:15:38.613258 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:15:38.613247    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls podName:cf58f785-a232-4130-bcc8-09226178623a nodeName:}" failed. No retries permitted until 2026-09-18 18:15:39.11322474 +0000 UTC m=+922.540437487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls") pod "maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" (UID: "cf58f785-a232-4130-bcc8-09226178623a") : secret "maas-api-e2e-ait-89dea9e1-serving-cert" not found
Sep 18 18:15:38.613423 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.613279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p556\" (UniqueName: \"kubernetes.io/projected/cf58f785-a232-4130-bcc8-09226178623a-kube-api-access-8p556\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.613423 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.613298    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf58f785-a232-4130-bcc8-09226178623a-openshift-service-ca\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.613934 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.613916    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf58f785-a232-4130-bcc8-09226178623a-openshift-service-ca\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.615898 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.615876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-metrics-certs\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:38.622974 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:38.622947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p556\" (UniqueName: \"kubernetes.io/projected/cf58f785-a232-4130-bcc8-09226178623a-kube-api-access-8p556\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:39.117898 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:39.117861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:39.125064 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:39.125023    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls\") pod \"maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:39.310194 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:39.310159    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:39.604344 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:39.604309    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"]
Sep 18 18:15:39.604983 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:15:39.604959    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf58f785_a232_4130_bcc8_09226178623a.slice/crio-dd13bf22ebc723d5e05b1b4a386d73a97067d919c8bda3b7ca06d99be129e992 WatchSource:0}: Error finding container dd13bf22ebc723d5e05b1b4a386d73a97067d919c8bda3b7ca06d99be129e992: Status 404 returned error can't find the container with id dd13bf22ebc723d5e05b1b4a386d73a97067d919c8bda3b7ca06d99be129e992
Sep 18 18:15:40.278663 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:40.278626    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" event={"ID":"cf58f785-a232-4130-bcc8-09226178623a","Type":"ContainerStarted","Data":"dd13bf22ebc723d5e05b1b4a386d73a97067d919c8bda3b7ca06d99be129e992"}
Sep 18 18:15:41.286958 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:41.286917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" event={"ID":"cf58f785-a232-4130-bcc8-09226178623a","Type":"ContainerStarted","Data":"097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056"}
Sep 18 18:15:41.287452 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:41.286993    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:15:41.306143 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:41.306094    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" podStartSLOduration=1.910008167 podStartE2EDuration="3.306078877s" totalImagesPullingTime="1.39607071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:41.303004691 +0000 UTC m=+924.730217444" watchObservedRunningTime="2026-09-18 18:15:41.306078877 +0000 UTC m=+924.733291628"
Sep 18 18:15:47.298617 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:15:47.298587    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:16:09.399584 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.399538    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"]
Sep 18 18:16:09.401764 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.399773    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" podUID="cf58f785-a232-4130-bcc8-09226178623a" containerName="maas-api" containerID="cri-o://097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056" gracePeriod=30
Sep 18 18:16:09.654188 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.654121    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:16:09.718624 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.718596    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-metrics-certs\") pod \"cf58f785-a232-4130-bcc8-09226178623a\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") "
Sep 18 18:16:09.718781 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.718651    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p556\" (UniqueName: \"kubernetes.io/projected/cf58f785-a232-4130-bcc8-09226178623a-kube-api-access-8p556\") pod \"cf58f785-a232-4130-bcc8-09226178623a\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") "
Sep 18 18:16:09.718781 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.718707    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf58f785-a232-4130-bcc8-09226178623a-openshift-service-ca\") pod \"cf58f785-a232-4130-bcc8-09226178623a\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") "
Sep 18 18:16:09.718892 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.718811    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls\") pod \"cf58f785-a232-4130-bcc8-09226178623a\" (UID: \"cf58f785-a232-4130-bcc8-09226178623a\") "
Sep 18 18:16:09.719203 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.719171    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf58f785-a232-4130-bcc8-09226178623a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf58f785-a232-4130-bcc8-09226178623a" (UID: "cf58f785-a232-4130-bcc8-09226178623a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:16:09.721217 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.721185    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf58f785-a232-4130-bcc8-09226178623a-kube-api-access-8p556" (OuterVolumeSpecName: "kube-api-access-8p556") pod "cf58f785-a232-4130-bcc8-09226178623a" (UID: "cf58f785-a232-4130-bcc8-09226178623a"). InnerVolumeSpecName "kube-api-access-8p556". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:16:09.721685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.721656    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf58f785-a232-4130-bcc8-09226178623a" (UID: "cf58f785-a232-4130-bcc8-09226178623a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:16:09.721685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.721667    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf58f785-a232-4130-bcc8-09226178623a" (UID: "cf58f785-a232-4130-bcc8-09226178623a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:16:09.820275 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.820240    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:16:09.820275 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.820268    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p556\" (UniqueName: \"kubernetes.io/projected/cf58f785-a232-4130-bcc8-09226178623a-kube-api-access-8p556\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:16:09.820275 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.820277    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf58f785-a232-4130-bcc8-09226178623a-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:16:09.820526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:09.820286    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf58f785-a232-4130-bcc8-09226178623a-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:16:10.485154 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.485115    2582 generic.go:356] "Generic (PLEG): container finished" podID="cf58f785-a232-4130-bcc8-09226178623a" containerID="097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056" exitCode=0
Sep 18 18:16:10.485601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.485171    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"
Sep 18 18:16:10.485601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.485194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" event={"ID":"cf58f785-a232-4130-bcc8-09226178623a","Type":"ContainerDied","Data":"097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056"}
Sep 18 18:16:10.485601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.485247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl" event={"ID":"cf58f785-a232-4130-bcc8-09226178623a","Type":"ContainerDied","Data":"dd13bf22ebc723d5e05b1b4a386d73a97067d919c8bda3b7ca06d99be129e992"}
Sep 18 18:16:10.485601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.485269    2582 scope.go:117] "RemoveContainer" containerID="097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056"
Sep 18 18:16:10.497797 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.497775    2582 scope.go:117] "RemoveContainer" containerID="097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056"
Sep 18 18:16:10.498106 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:16:10.498081    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056\": container with ID starting with 097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056 not found: ID does not exist" containerID="097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056"
Sep 18 18:16:10.498188 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.498116    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056"} err="failed to get container status \"097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056\": rpc error: code = NotFound desc = could not find container \"097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056\": container with ID starting with 097f11fef5b9066f5ca6c45ee4b3d70d4a38ec3683a455c1099b5e99c45fb056 not found: ID does not exist"
Sep 18 18:16:10.510941 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.510920    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"]
Sep 18 18:16:10.516518 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.516500    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89dea9e1-69cb944b6b-qfshl"]
Sep 18 18:16:10.921340 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:10.921308    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf58f785-a232-4130-bcc8-09226178623a" path="/var/lib/kubelet/pods/cf58f785-a232-4130-bcc8-09226178623a/volumes"
Sep 18 18:16:39.446935 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.446904    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"]
Sep 18 18:16:39.447553 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.447532    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf58f785-a232-4130-bcc8-09226178623a" containerName="maas-api"
Sep 18 18:16:39.447608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.447559    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf58f785-a232-4130-bcc8-09226178623a" containerName="maas-api"
Sep 18 18:16:39.447709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.447698    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf58f785-a232-4130-bcc8-09226178623a" containerName="maas-api"
Sep 18 18:16:39.451371 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.451351    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:39.454572 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.454539    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:16:39.458126 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.458093    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"]
Sep 18 18:16:39.510177 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.510148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/615fd959-a7ac-445a-8dc1-d5d199ad89eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-94929d65-tzzzx\" (UID: \"615fd959-a7ac-445a-8dc1-d5d199ad89eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:39.611459 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.611427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/615fd959-a7ac-445a-8dc1-d5d199ad89eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-94929d65-tzzzx\" (UID: \"615fd959-a7ac-445a-8dc1-d5d199ad89eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:39.612015 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.611998    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/615fd959-a7ac-445a-8dc1-d5d199ad89eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-94929d65-tzzzx\" (UID: \"615fd959-a7ac-445a-8dc1-d5d199ad89eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:39.769979 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:39.769900    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:40.046355 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:40.046323    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"]
Sep 18 18:16:40.047585 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:16:40.047549    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615fd959_a7ac_445a_8dc1_d5d199ad89eb.slice/crio-d89392e616a784a2ab301fbe6c3c047c5e613bd0d051064e2c5952f1356fa764 WatchSource:0}: Error finding container d89392e616a784a2ab301fbe6c3c047c5e613bd0d051064e2c5952f1356fa764: Status 404 returned error can't find the container with id d89392e616a784a2ab301fbe6c3c047c5e613bd0d051064e2c5952f1356fa764
Sep 18 18:16:40.683703 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:40.683612    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx" event={"ID":"615fd959-a7ac-445a-8dc1-d5d199ad89eb","Type":"ContainerStarted","Data":"cb1e99ad2e1f661ca39edc80fd2e89a4569aaa49c9b9911f244312b7d52c66f5"}
Sep 18 18:16:40.683703 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:40.683663    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx" event={"ID":"615fd959-a7ac-445a-8dc1-d5d199ad89eb","Type":"ContainerStarted","Data":"d89392e616a784a2ab301fbe6c3c047c5e613bd0d051064e2c5952f1356fa764"}
Sep 18 18:16:40.699075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:40.699007    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx" podStartSLOduration=1.6989923679999999 podStartE2EDuration="1.698992368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:16:40.697129292 +0000 UTC m=+984.124342047" watchObservedRunningTime="2026-09-18 18:16:40.698992368 +0000 UTC m=+984.126205121"
Sep 18 18:16:41.691361 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:41.691322    2582 generic.go:356] "Generic (PLEG): container finished" podID="615fd959-a7ac-445a-8dc1-d5d199ad89eb" containerID="cb1e99ad2e1f661ca39edc80fd2e89a4569aaa49c9b9911f244312b7d52c66f5" exitCode=0
Sep 18 18:16:41.691744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:41.691406    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx" event={"ID":"615fd959-a7ac-445a-8dc1-d5d199ad89eb","Type":"ContainerDied","Data":"cb1e99ad2e1f661ca39edc80fd2e89a4569aaa49c9b9911f244312b7d52c66f5"}
Sep 18 18:16:42.839777 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:42.839756    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:42.947184 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:42.947152    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/615fd959-a7ac-445a-8dc1-d5d199ad89eb-maas-api-service-ca\") pod \"615fd959-a7ac-445a-8dc1-d5d199ad89eb\" (UID: \"615fd959-a7ac-445a-8dc1-d5d199ad89eb\") "
Sep 18 18:16:42.947504 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:42.947479    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615fd959-a7ac-445a-8dc1-d5d199ad89eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "615fd959-a7ac-445a-8dc1-d5d199ad89eb" (UID: "615fd959-a7ac-445a-8dc1-d5d199ad89eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:16:43.048559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:43.048531    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/615fd959-a7ac-445a-8dc1-d5d199ad89eb-maas-api-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:16:43.705186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:43.705158    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"
Sep 18 18:16:43.705328 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:43.705205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx" event={"ID":"615fd959-a7ac-445a-8dc1-d5d199ad89eb","Type":"ContainerDied","Data":"d89392e616a784a2ab301fbe6c3c047c5e613bd0d051064e2c5952f1356fa764"}
Sep 18 18:16:43.705328 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:43.705251    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d89392e616a784a2ab301fbe6c3c047c5e613bd0d051064e2c5952f1356fa764"
Sep 18 18:16:46.790697 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:46.790664    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"]
Sep 18 18:16:46.795107 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:46.795069    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-94929d65-tzzzx"]
Sep 18 18:16:46.925893 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:16:46.925854    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615fd959-a7ac-445a-8dc1-d5d199ad89eb" path="/var/lib/kubelet/pods/615fd959-a7ac-445a-8dc1-d5d199ad89eb/volumes"
Sep 18 18:18:28.790218 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.790138    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"]
Sep 18 18:18:28.790852 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.790835    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615fd959-a7ac-445a-8dc1-d5d199ad89eb" containerName="revoke-keys"
Sep 18 18:18:28.790907 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.790856    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="615fd959-a7ac-445a-8dc1-d5d199ad89eb" containerName="revoke-keys"
Sep 18 18:18:28.791017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.791006    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="615fd959-a7ac-445a-8dc1-d5d199ad89eb" containerName="revoke-keys"
Sep 18 18:18:28.795430 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.795408    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:28.797787 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.797765    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-026961cf-serving-cert\""
Sep 18 18:18:28.798608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.798589    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:18:28.798755 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.798734    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jkbfn\""
Sep 18 18:18:28.805191 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.804899    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"]
Sep 18 18:18:28.943230 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.943195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6tw\" (UniqueName: \"kubernetes.io/projected/6888114d-93f0-47e1-8d33-72070f508e17-kube-api-access-xg6tw\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:28.943230 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.943231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-metrics-certs\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:28.943592 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.943260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-maas-api-tls\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:28.943592 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:28.943279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6888114d-93f0-47e1-8d33-72070f508e17-openshift-service-ca\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.044936 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.044847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6tw\" (UniqueName: \"kubernetes.io/projected/6888114d-93f0-47e1-8d33-72070f508e17-kube-api-access-xg6tw\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.044936 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.044885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-metrics-certs\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.044936 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.044921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-maas-api-tls\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.045217 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.044941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6888114d-93f0-47e1-8d33-72070f508e17-openshift-service-ca\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.045669 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.045644    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6888114d-93f0-47e1-8d33-72070f508e17-openshift-service-ca\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.047490 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.047465    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-metrics-certs\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.047630 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.047610    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-maas-api-tls\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.054921 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.054903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6tw\" (UniqueName: \"kubernetes.io/projected/6888114d-93f0-47e1-8d33-72070f508e17-kube-api-access-xg6tw\") pod \"maas-api-e2e-mt-026961cf-96d874966-tmvqp\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.109821 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.109789    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:29.392164 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.391998    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"]
Sep 18 18:18:29.393238 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:18:29.393203    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6888114d_93f0_47e1_8d33_72070f508e17.slice/crio-dc83c179cc7067c0ccfb41b997fbe14c535056c68acb9c9616a8367a493a679d WatchSource:0}: Error finding container dc83c179cc7067c0ccfb41b997fbe14c535056c68acb9c9616a8367a493a679d: Status 404 returned error can't find the container with id dc83c179cc7067c0ccfb41b997fbe14c535056c68acb9c9616a8367a493a679d
Sep 18 18:18:29.394907 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:29.394888    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:18:30.394912 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:30.394869    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp" event={"ID":"6888114d-93f0-47e1-8d33-72070f508e17","Type":"ContainerStarted","Data":"387411d633da500814a92d626368e89686894050f909e5d5bf6ad120623465da"}
Sep 18 18:18:30.395386 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:30.394944    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:18:30.395386 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:30.394966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp" event={"ID":"6888114d-93f0-47e1-8d33-72070f508e17","Type":"ContainerStarted","Data":"dc83c179cc7067c0ccfb41b997fbe14c535056c68acb9c9616a8367a493a679d"}
Sep 18 18:18:30.412247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:30.412200    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp" podStartSLOduration=1.9504157100000001 podStartE2EDuration="2.412185084s" totalImagesPullingTime="461.769374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:18:30.411521389 +0000 UTC m=+1093.838734145" watchObservedRunningTime="2026-09-18 18:18:30.412185084 +0000 UTC m=+1093.839397887"
Sep 18 18:18:36.407531 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:18:36.407502    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:19:08.515618 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.515588    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"]
Sep 18 18:19:08.516259 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.515827    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp" podUID="6888114d-93f0-47e1-8d33-72070f508e17" containerName="maas-api" containerID="cri-o://387411d633da500814a92d626368e89686894050f909e5d5bf6ad120623465da" gracePeriod=30
Sep 18 18:19:08.650620 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.650587    2582 generic.go:356] "Generic (PLEG): container finished" podID="6888114d-93f0-47e1-8d33-72070f508e17" containerID="387411d633da500814a92d626368e89686894050f909e5d5bf6ad120623465da" exitCode=0
Sep 18 18:19:08.650784 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.650660    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp" event={"ID":"6888114d-93f0-47e1-8d33-72070f508e17","Type":"ContainerDied","Data":"387411d633da500814a92d626368e89686894050f909e5d5bf6ad120623465da"}
Sep 18 18:19:08.771186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.771126    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:19:08.825463 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.825435    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6888114d-93f0-47e1-8d33-72070f508e17-openshift-service-ca\") pod \"6888114d-93f0-47e1-8d33-72070f508e17\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") "
Sep 18 18:19:08.825608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.825471    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-metrics-certs\") pod \"6888114d-93f0-47e1-8d33-72070f508e17\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") "
Sep 18 18:19:08.825651 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.825602    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-maas-api-tls\") pod \"6888114d-93f0-47e1-8d33-72070f508e17\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") "
Sep 18 18:19:08.825687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.825652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg6tw\" (UniqueName: \"kubernetes.io/projected/6888114d-93f0-47e1-8d33-72070f508e17-kube-api-access-xg6tw\") pod \"6888114d-93f0-47e1-8d33-72070f508e17\" (UID: \"6888114d-93f0-47e1-8d33-72070f508e17\") "
Sep 18 18:19:08.825856 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.825831    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6888114d-93f0-47e1-8d33-72070f508e17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6888114d-93f0-47e1-8d33-72070f508e17" (UID: "6888114d-93f0-47e1-8d33-72070f508e17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:08.826166 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.826143    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6888114d-93f0-47e1-8d33-72070f508e17-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:08.827914 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.827890    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6888114d-93f0-47e1-8d33-72070f508e17" (UID: "6888114d-93f0-47e1-8d33-72070f508e17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:19:08.828019 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.827928    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6888114d-93f0-47e1-8d33-72070f508e17" (UID: "6888114d-93f0-47e1-8d33-72070f508e17"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:19:08.828019 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.827934    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6888114d-93f0-47e1-8d33-72070f508e17-kube-api-access-xg6tw" (OuterVolumeSpecName: "kube-api-access-xg6tw") pod "6888114d-93f0-47e1-8d33-72070f508e17" (UID: "6888114d-93f0-47e1-8d33-72070f508e17"). InnerVolumeSpecName "kube-api-access-xg6tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:08.927784 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.927730    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:08.927784 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.927778    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6888114d-93f0-47e1-8d33-72070f508e17-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:08.928086 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:08.927792    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg6tw\" (UniqueName: \"kubernetes.io/projected/6888114d-93f0-47e1-8d33-72070f508e17-kube-api-access-xg6tw\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.657858 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:09.657823    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"
Sep 18 18:19:09.658349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:09.657869    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp" event={"ID":"6888114d-93f0-47e1-8d33-72070f508e17","Type":"ContainerDied","Data":"dc83c179cc7067c0ccfb41b997fbe14c535056c68acb9c9616a8367a493a679d"}
Sep 18 18:19:09.658349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:09.657942    2582 scope.go:117] "RemoveContainer" containerID="387411d633da500814a92d626368e89686894050f909e5d5bf6ad120623465da"
Sep 18 18:19:09.679467 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:09.679439    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"]
Sep 18 18:19:09.683502 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:09.683480    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-026961cf-96d874966-tmvqp"]
Sep 18 18:19:10.923237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:10.923203    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6888114d-93f0-47e1-8d33-72070f508e17" path="/var/lib/kubelet/pods/6888114d-93f0-47e1-8d33-72070f508e17/volumes"
Sep 18 18:19:34.690238 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.690203    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"]
Sep 18 18:19:34.690674 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.690659    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6888114d-93f0-47e1-8d33-72070f508e17" containerName="maas-api"
Sep 18 18:19:34.690721 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.690675    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6888114d-93f0-47e1-8d33-72070f508e17" containerName="maas-api"
Sep 18 18:19:34.690769 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.690759    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6888114d-93f0-47e1-8d33-72070f508e17" containerName="maas-api"
Sep 18 18:19:34.693942 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.693924    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:34.697156 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.697132    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:19:34.700126 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.700096    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"]
Sep 18 18:19:34.779691 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.779659    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/512aaacd-0735-45d4-8c04-53c6575a1122-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp\" (UID: \"512aaacd-0735-45d4-8c04-53c6575a1122\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:34.880257 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.880218    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/512aaacd-0735-45d4-8c04-53c6575a1122-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp\" (UID: \"512aaacd-0735-45d4-8c04-53c6575a1122\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:34.880878 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:34.880858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/512aaacd-0735-45d4-8c04-53c6575a1122-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp\" (UID: \"512aaacd-0735-45d4-8c04-53c6575a1122\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:35.008598 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.008523    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:35.204684 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.204647    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"]
Sep 18 18:19:35.208901 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.208879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.211426 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.211403    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jkbfn\""
Sep 18 18:19:35.211559 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.211452    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-f5e6d4-serving-cert\""
Sep 18 18:19:35.212117 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.212094    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:19:35.225090 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.225026    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"]
Sep 18 18:19:35.282996 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.282963    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"]
Sep 18 18:19:35.283674 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:19:35.283650    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512aaacd_0735_45d4_8c04_53c6575a1122.slice/crio-0647ed28de4144410c856b12b74755d6050a26a53cefb8d6399469ebf8c9faba WatchSource:0}: Error finding container 0647ed28de4144410c856b12b74755d6050a26a53cefb8d6399469ebf8c9faba: Status 404 returned error can't find the container with id 0647ed28de4144410c856b12b74755d6050a26a53cefb8d6399469ebf8c9faba
Sep 18 18:19:35.285718 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.285693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.285799 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.285747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad27a2d-eefe-4b43-b0ae-1cde44277917-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.285847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.285796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v85t6\" (UniqueName: \"kubernetes.io/projected/fad27a2d-eefe-4b43-b0ae-1cde44277917-kube-api-access-v85t6\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.285847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.285823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.386835 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.386806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v85t6\" (UniqueName: \"kubernetes.io/projected/fad27a2d-eefe-4b43-b0ae-1cde44277917-kube-api-access-v85t6\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.386835 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.386839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.387027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.386939    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.387027 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.386983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad27a2d-eefe-4b43-b0ae-1cde44277917-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.387154 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:19:35.387130    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-serving-cert: secret "maas-api-e2e-shared-a-w1-f5e6d4-serving-cert" not found
Sep 18 18:19:35.387212 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:19:35.387203    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls podName:fad27a2d-eefe-4b43-b0ae-1cde44277917 nodeName:}" failed. No retries permitted until 2026-09-18 18:19:35.88718468 +0000 UTC m=+1159.314397422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls") pod "maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" (UID: "fad27a2d-eefe-4b43-b0ae-1cde44277917") : secret "maas-api-e2e-shared-a-w1-f5e6d4-serving-cert" not found
Sep 18 18:19:35.387736 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.387664    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad27a2d-eefe-4b43-b0ae-1cde44277917-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.389679 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.389654    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.395142 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.395118    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v85t6\" (UniqueName: \"kubernetes.io/projected/fad27a2d-eefe-4b43-b0ae-1cde44277917-kube-api-access-v85t6\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.842582 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.842489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp" event={"ID":"512aaacd-0735-45d4-8c04-53c6575a1122","Type":"ContainerStarted","Data":"0812d172f39421bf3355fde7e4c07ac2bd62684d10efa81adc30858809d048c0"}
Sep 18 18:19:35.842582 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.842556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp" event={"ID":"512aaacd-0735-45d4-8c04-53c6575a1122","Type":"ContainerStarted","Data":"0647ed28de4144410c856b12b74755d6050a26a53cefb8d6399469ebf8c9faba"}
Sep 18 18:19:35.865155 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.865107    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp" podStartSLOduration=1.865090694 podStartE2EDuration="1.865090694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:19:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:19:35.861457813 +0000 UTC m=+1159.288670570" watchObservedRunningTime="2026-09-18 18:19:35.865090694 +0000 UTC m=+1159.292303451"
Sep 18 18:19:35.893282 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.893249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:35.895746 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:35.895727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:36.126302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:36.126227    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:36.406414 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:36.406387    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"]
Sep 18 18:19:36.408317 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:19:36.408289    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad27a2d_eefe_4b43_b0ae_1cde44277917.slice/crio-54dd46feb19c0494ac2037e226df33f4b06356aa6bddca6b1ff2744d75d2001d WatchSource:0}: Error finding container 54dd46feb19c0494ac2037e226df33f4b06356aa6bddca6b1ff2744d75d2001d: Status 404 returned error can't find the container with id 54dd46feb19c0494ac2037e226df33f4b06356aa6bddca6b1ff2744d75d2001d
Sep 18 18:19:36.851594 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:36.851566    2582 generic.go:356] "Generic (PLEG): container finished" podID="512aaacd-0735-45d4-8c04-53c6575a1122" containerID="0812d172f39421bf3355fde7e4c07ac2bd62684d10efa81adc30858809d048c0" exitCode=0
Sep 18 18:19:36.851949 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:36.851646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp" event={"ID":"512aaacd-0735-45d4-8c04-53c6575a1122","Type":"ContainerDied","Data":"0812d172f39421bf3355fde7e4c07ac2bd62684d10efa81adc30858809d048c0"}
Sep 18 18:19:36.853071 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:36.853021    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" event={"ID":"fad27a2d-eefe-4b43-b0ae-1cde44277917","Type":"ContainerStarted","Data":"54dd46feb19c0494ac2037e226df33f4b06356aa6bddca6b1ff2744d75d2001d"}
Sep 18 18:19:37.861175 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:37.861142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" event={"ID":"fad27a2d-eefe-4b43-b0ae-1cde44277917","Type":"ContainerStarted","Data":"49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2"}
Sep 18 18:19:37.881829 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:37.881779    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" podStartSLOduration=2.467449623 podStartE2EDuration="2.88176199s" totalImagesPullingTime="414.312367ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:19:37.879087071 +0000 UTC m=+1161.306299827" watchObservedRunningTime="2026-09-18 18:19:37.88176199 +0000 UTC m=+1161.308974744"
Sep 18 18:19:38.010580 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.010555    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:38.117162 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.117080    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/512aaacd-0735-45d4-8c04-53c6575a1122-maas-api-service-ca\") pod \"512aaacd-0735-45d4-8c04-53c6575a1122\" (UID: \"512aaacd-0735-45d4-8c04-53c6575a1122\") "
Sep 18 18:19:38.117488 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.117466    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/512aaacd-0735-45d4-8c04-53c6575a1122-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "512aaacd-0735-45d4-8c04-53c6575a1122" (UID: "512aaacd-0735-45d4-8c04-53c6575a1122"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:38.218472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.218436    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/512aaacd-0735-45d4-8c04-53c6575a1122-maas-api-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:38.869455 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.869423    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"
Sep 18 18:19:38.869860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.869468    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp" event={"ID":"512aaacd-0735-45d4-8c04-53c6575a1122","Type":"ContainerDied","Data":"0647ed28de4144410c856b12b74755d6050a26a53cefb8d6399469ebf8c9faba"}
Sep 18 18:19:38.869860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.869516    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0647ed28de4144410c856b12b74755d6050a26a53cefb8d6399469ebf8c9faba"
Sep 18 18:19:38.869860 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:38.869839    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:44.884779 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:44.884747    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:19:45.429124 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.429091    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"]
Sep 18 18:19:45.429377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.429352    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9" podUID="a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" containerName="payload-pre-processing" containerID="cri-o://1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c" gracePeriod=30
Sep 18 18:19:45.645537 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.645503    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"]
Sep 18 18:19:45.653486 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.653456    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-de7535-wv8sp"]
Sep 18 18:19:45.716508 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.716483    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:19:45.793847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.793812    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-plugins-config-volume\") pod \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") "
Sep 18 18:19:45.794010 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.793932    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq9kb\" (UniqueName: \"kubernetes.io/projected/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-kube-api-access-xq9kb\") pod \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\" (UID: \"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c\") "
Sep 18 18:19:45.794232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.794207    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" (UID: "a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:45.796508 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.796465    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-kube-api-access-xq9kb" (OuterVolumeSpecName: "kube-api-access-xq9kb") pod "a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" (UID: "a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c"). InnerVolumeSpecName "kube-api-access-xq9kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:45.895469 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.895437    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:45.895469 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.895464    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq9kb\" (UniqueName: \"kubernetes.io/projected/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c-kube-api-access-xq9kb\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:19:45.918253 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.918218    2582 generic.go:356] "Generic (PLEG): container finished" podID="a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" containerID="1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c" exitCode=0
Sep 18 18:19:45.918383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.918256    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"
Sep 18 18:19:45.918383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.918296    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9" event={"ID":"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c","Type":"ContainerDied","Data":"1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c"}
Sep 18 18:19:45.918383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.918343    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9" event={"ID":"a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c","Type":"ContainerDied","Data":"af2953bec483b8d78e0183769ec9733404337fb88afb7eaa3f3691c82860129b"}
Sep 18 18:19:45.918383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.918366    2582 scope.go:117] "RemoveContainer" containerID="1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c"
Sep 18 18:19:45.929546 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.929522    2582 scope.go:117] "RemoveContainer" containerID="1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c"
Sep 18 18:19:45.929864 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:19:45.929833    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c\": container with ID starting with 1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c not found: ID does not exist" containerID="1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c"
Sep 18 18:19:45.929942 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.929865    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c"} err="failed to get container status \"1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c\": rpc error: code = NotFound desc = could not find container \"1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c\": container with ID starting with 1b37cf1ff0842f32fb097c9791e876fbda393431c55265a19198c567e9dc9d2c not found: ID does not exist"
Sep 18 18:19:45.945287 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.945228    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"]
Sep 18 18:19:45.950709 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:45.950686    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-de7535-c9768d786-f77j9"]
Sep 18 18:19:46.920341 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:46.920304    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="512aaacd-0735-45d4-8c04-53c6575a1122" path="/var/lib/kubelet/pods/512aaacd-0735-45d4-8c04-53c6575a1122/volumes"
Sep 18 18:19:46.920837 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:46.920756    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" path="/var/lib/kubelet/pods/a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c/volumes"
Sep 18 18:19:57.189731 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.189694    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"]
Sep 18 18:19:57.190208 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.190177    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" containerName="payload-pre-processing"
Sep 18 18:19:57.190208 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.190190    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" containerName="payload-pre-processing"
Sep 18 18:19:57.190208 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.190209    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="512aaacd-0735-45d4-8c04-53c6575a1122" containerName="revoke-keys"
Sep 18 18:19:57.190381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.190214    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="512aaacd-0735-45d4-8c04-53c6575a1122" containerName="revoke-keys"
Sep 18 18:19:57.190381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.190289    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="512aaacd-0735-45d4-8c04-53c6575a1122" containerName="revoke-keys"
Sep 18 18:19:57.190381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.190297    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b1dbfb-f6a9-43aa-b26a-c0cbfb40234c" containerName="payload-pre-processing"
Sep 18 18:19:57.194842 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.194821    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.197413 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.197390    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-1f7000-serving-cert\""
Sep 18 18:19:57.206319 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.206293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"]
Sep 18 18:19:57.308491 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.308463    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611a5f86-321e-4ab3-a32c-95237850af4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.308652 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.308523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.308698 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.308672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.308796 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.308732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crqfp\" (UniqueName: \"kubernetes.io/projected/611a5f86-321e-4ab3-a32c-95237850af4c-kube-api-access-crqfp\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.410315 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.410279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.410459 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.410393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.410459 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.410443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crqfp\" (UniqueName: \"kubernetes.io/projected/611a5f86-321e-4ab3-a32c-95237850af4c-kube-api-access-crqfp\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.410569 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.410495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611a5f86-321e-4ab3-a32c-95237850af4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.411121 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.411094    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611a5f86-321e-4ab3-a32c-95237850af4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.412859 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.412840    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.412979 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.412915    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.418993 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.418972    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crqfp\" (UniqueName: \"kubernetes.io/projected/611a5f86-321e-4ab3-a32c-95237850af4c-kube-api-access-crqfp\") pod \"maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.509420 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.509351    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:57.791207 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.791178    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"]
Sep 18 18:19:57.792624 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:19:57.792600    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611a5f86_321e_4ab3_a32c_95237850af4c.slice/crio-0dba83d413cceada63355b20aba6f21650aa08056429777ecbb51dac4d10e657 WatchSource:0}: Error finding container 0dba83d413cceada63355b20aba6f21650aa08056429777ecbb51dac4d10e657: Status 404 returned error can't find the container with id 0dba83d413cceada63355b20aba6f21650aa08056429777ecbb51dac4d10e657
Sep 18 18:19:57.998002 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:57.997966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5" event={"ID":"611a5f86-321e-4ab3-a32c-95237850af4c","Type":"ContainerStarted","Data":"0dba83d413cceada63355b20aba6f21650aa08056429777ecbb51dac4d10e657"}
Sep 18 18:19:59.005221 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:59.005183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5" event={"ID":"611a5f86-321e-4ab3-a32c-95237850af4c","Type":"ContainerStarted","Data":"256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92"}
Sep 18 18:19:59.005623 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:59.005255    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:19:59.027230 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:19:59.027179    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5" podStartSLOduration=1.65467607 podStartE2EDuration="2.027161648s" totalImagesPullingTime="372.485578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:19:59.023232479 +0000 UTC m=+1182.450445236" watchObservedRunningTime="2026-09-18 18:19:59.027161648 +0000 UTC m=+1182.454374404"
Sep 18 18:20:03.885399 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:03.885367    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"]
Sep 18 18:20:03.888310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:03.888276    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-bcf02f-29829255-zsljf"]
Sep 18 18:20:03.915636 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:03.915604    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"]
Sep 18 18:20:03.915843 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:03.915814    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424" podUID="89bac72a-3a9d-4586-9d27-5a87cfa5ac57" containerName="payload-processing" containerID="cri-o://0d9d0290a3dd8cdb4aaaaccdda88eaf67bcbae730865ba9b58217981d1bc069a" gracePeriod=30
Sep 18 18:20:04.039342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.039310    2582 generic.go:356] "Generic (PLEG): container finished" podID="89bac72a-3a9d-4586-9d27-5a87cfa5ac57" containerID="0d9d0290a3dd8cdb4aaaaccdda88eaf67bcbae730865ba9b58217981d1bc069a" exitCode=0
Sep 18 18:20:04.039489 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.039386    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424" event={"ID":"89bac72a-3a9d-4586-9d27-5a87cfa5ac57","Type":"ContainerDied","Data":"0d9d0290a3dd8cdb4aaaaccdda88eaf67bcbae730865ba9b58217981d1bc069a"}
Sep 18 18:20:04.189247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.189220    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:20:04.281255 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.281222    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-plugins-config-volume\") pod \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") "
Sep 18 18:20:04.281451 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.281269    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdbk5\" (UniqueName: \"kubernetes.io/projected/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-kube-api-access-wdbk5\") pod \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\" (UID: \"89bac72a-3a9d-4586-9d27-5a87cfa5ac57\") "
Sep 18 18:20:04.281652 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.281625    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89bac72a-3a9d-4586-9d27-5a87cfa5ac57" (UID: "89bac72a-3a9d-4586-9d27-5a87cfa5ac57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:20:04.283590 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.283565    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-kube-api-access-wdbk5" (OuterVolumeSpecName: "kube-api-access-wdbk5") pod "89bac72a-3a9d-4586-9d27-5a87cfa5ac57" (UID: "89bac72a-3a9d-4586-9d27-5a87cfa5ac57"). InnerVolumeSpecName "kube-api-access-wdbk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:04.382894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.382868    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:20:04.382894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.382891    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdbk5\" (UniqueName: \"kubernetes.io/projected/89bac72a-3a9d-4586-9d27-5a87cfa5ac57-kube-api-access-wdbk5\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:20:04.921110 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:04.921077    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1" path="/var/lib/kubelet/pods/8e4e7c0d-2dc9-4741-ae55-9d5093cf8db1/volumes"
Sep 18 18:20:05.017752 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:05.017718    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:20:05.048564 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:05.048536    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"
Sep 18 18:20:05.048564 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:05.048541    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424" event={"ID":"89bac72a-3a9d-4586-9d27-5a87cfa5ac57","Type":"ContainerDied","Data":"f052263b97c0ae2316836ac38641542d39c166607482668328285abbceb902a7"}
Sep 18 18:20:05.048783 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:05.048611    2582 scope.go:117] "RemoveContainer" containerID="0d9d0290a3dd8cdb4aaaaccdda88eaf67bcbae730865ba9b58217981d1bc069a"
Sep 18 18:20:05.069730 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:05.069701    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"]
Sep 18 18:20:05.074526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:05.074495    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bcf02f-67b6566689-vv424"]
Sep 18 18:20:06.920716 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:06.920685    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89bac72a-3a9d-4586-9d27-5a87cfa5ac57" path="/var/lib/kubelet/pods/89bac72a-3a9d-4586-9d27-5a87cfa5ac57/volumes"
Sep 18 18:20:10.716993 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.716958    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8894-e7d1ab"]
Sep 18 18:20:10.717448 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.717433    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89bac72a-3a9d-4586-9d27-5a87cfa5ac57" containerName="payload-processing"
Sep 18 18:20:10.717448 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.717449    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="89bac72a-3a9d-4586-9d27-5a87cfa5ac57" containerName="payload-processing"
Sep 18 18:20:10.717529 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.717520    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="89bac72a-3a9d-4586-9d27-5a87cfa5ac57" containerName="payload-processing"
Sep 18 18:20:10.722115 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.722091    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:10.724353 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.724332    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxj55\""
Sep 18 18:20:10.725015 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.724987    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-e7d1ab"]
Sep 18 18:20:10.747820 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.747791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svps8\" (UniqueName: \"kubernetes.io/projected/181dd568-2673-442f-8f8f-bf84c9e1b969-kube-api-access-svps8\") pod \"test-curl-8894-e7d1ab\" (UID: \"181dd568-2673-442f-8f8f-bf84c9e1b969\") " pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:10.848574 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.848538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svps8\" (UniqueName: \"kubernetes.io/projected/181dd568-2673-442f-8f8f-bf84c9e1b969-kube-api-access-svps8\") pod \"test-curl-8894-e7d1ab\" (UID: \"181dd568-2673-442f-8f8f-bf84c9e1b969\") " pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:10.857383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:10.857360    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svps8\" (UniqueName: \"kubernetes.io/projected/181dd568-2673-442f-8f8f-bf84c9e1b969-kube-api-access-svps8\") pod \"test-curl-8894-e7d1ab\" (UID: \"181dd568-2673-442f-8f8f-bf84c9e1b969\") " pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:11.034133 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:11.034106    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:11.311250 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:11.311217    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-e7d1ab"]
Sep 18 18:20:11.312737 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:11.312711    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod181dd568_2673_442f_8f8f_bf84c9e1b969.slice/crio-02d8a38f2d963a2fd0842cb98af72a3def387cc83c819f97c73b81826172609b WatchSource:0}: Error finding container 02d8a38f2d963a2fd0842cb98af72a3def387cc83c819f97c73b81826172609b: Status 404 returned error can't find the container with id 02d8a38f2d963a2fd0842cb98af72a3def387cc83c819f97c73b81826172609b
Sep 18 18:20:12.098463 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.098430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-e7d1ab" event={"ID":"181dd568-2673-442f-8f8f-bf84c9e1b969","Type":"ContainerStarted","Data":"372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873"}
Sep 18 18:20:12.098833 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.098474    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-e7d1ab" event={"ID":"181dd568-2673-442f-8f8f-bf84c9e1b969","Type":"ContainerStarted","Data":"02d8a38f2d963a2fd0842cb98af72a3def387cc83c819f97c73b81826172609b"}
Sep 18 18:20:12.111858 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.111814    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8894-e7d1ab" podStartSLOduration=1.429871275 podStartE2EDuration="2.111799451s" totalImagesPullingTime="681.928176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:12.110673146 +0000 UTC m=+1195.537885900" watchObservedRunningTime="2026-09-18 18:20:12.111799451 +0000 UTC m=+1195.539012205"
Sep 18 18:20:12.727096 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.726981    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8894-e7d1ab"]
Sep 18 18:20:12.730130 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.730097    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8894-e7d1ab"]
Sep 18 18:20:12.844303 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.844263    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8894-2d9a54"]
Sep 18 18:20:12.844779 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.844764    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181dd568-2673-442f-8f8f-bf84c9e1b969" containerName="test-curl-8894-e7d1ab"
Sep 18 18:20:12.844837 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.844781    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="181dd568-2673-442f-8f8f-bf84c9e1b969" containerName="test-curl-8894-e7d1ab"
Sep 18 18:20:12.844879 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.844868    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="181dd568-2673-442f-8f8f-bf84c9e1b969" containerName="test-curl-8894-e7d1ab"
Sep 18 18:20:12.848141 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.848125    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:12.853101 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.853042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-2d9a54"]
Sep 18 18:20:12.868282 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.868258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcdt5\" (UniqueName: \"kubernetes.io/projected/f6a795d5-cc96-4b53-8ae7-740cc712d5a4-kube-api-access-wcdt5\") pod \"test-curl-8894-2d9a54\" (UID: \"f6a795d5-cc96-4b53-8ae7-740cc712d5a4\") " pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:12.969395 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.969363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcdt5\" (UniqueName: \"kubernetes.io/projected/f6a795d5-cc96-4b53-8ae7-740cc712d5a4-kube-api-access-wcdt5\") pod \"test-curl-8894-2d9a54\" (UID: \"f6a795d5-cc96-4b53-8ae7-740cc712d5a4\") " pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:12.977366 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:12.977290    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcdt5\" (UniqueName: \"kubernetes.io/projected/f6a795d5-cc96-4b53-8ae7-740cc712d5a4-kube-api-access-wcdt5\") pod \"test-curl-8894-2d9a54\" (UID: \"f6a795d5-cc96-4b53-8ae7-740cc712d5a4\") " pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:13.104734 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:13.104707    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8894-e7d1ab" container="test-curl-8894-e7d1ab"
Sep 18 18:20:13.159727 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:13.159692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:13.431233 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:13.431193    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-2d9a54"]
Sep 18 18:20:13.431382 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:13.431193    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce WatchSource:0}: Error finding container bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce: Status 404 returned error can't find the container with id bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce
Sep 18 18:20:14.113380 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.113341    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-2d9a54" event={"ID":"f6a795d5-cc96-4b53-8ae7-740cc712d5a4","Type":"ContainerStarted","Data":"1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41"}
Sep 18 18:20:14.113838 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.113390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-2d9a54" event={"ID":"f6a795d5-cc96-4b53-8ae7-740cc712d5a4","Type":"ContainerStarted","Data":"bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce"}
Sep 18 18:20:14.113838 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.113460    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8894-e7d1ab" podUID="181dd568-2673-442f-8f8f-bf84c9e1b969" containerName="test-curl-8894-e7d1ab" containerID="cri-o://372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873" gracePeriod=2
Sep 18 18:20:14.128910 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.128860    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8894-2d9a54" podStartSLOduration=2.128846157 podStartE2EDuration="2.128846157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:14.127041491 +0000 UTC m=+1197.554254246" watchObservedRunningTime="2026-09-18 18:20:14.128846157 +0000 UTC m=+1197.556058912"
Sep 18 18:20:14.848773 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.848726    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8894-2d9a54"]
Sep 18 18:20:14.852191 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.852119    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8894-2d9a54"]
Sep 18 18:20:14.966680 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.966646    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8894-394926"]
Sep 18 18:20:14.967152 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.967137    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6a795d5-cc96-4b53-8ae7-740cc712d5a4" containerName="test-curl-8894-2d9a54"
Sep 18 18:20:14.967210 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.967154    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6a795d5-cc96-4b53-8ae7-740cc712d5a4" containerName="test-curl-8894-2d9a54"
Sep 18 18:20:14.967243 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.967236    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6a795d5-cc96-4b53-8ae7-740cc712d5a4" containerName="test-curl-8894-2d9a54"
Sep 18 18:20:14.970446 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.970429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:14.976321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.976297    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-394926"]
Sep 18 18:20:14.989353 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:14.989328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29zp\" (UniqueName: \"kubernetes.io/projected/0452390d-e7b4-49f2-9501-a7db296d6f29-kube-api-access-n29zp\") pod \"test-curl-8894-394926\" (UID: \"0452390d-e7b4-49f2-9501-a7db296d6f29\") " pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:15.090803 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:15.090771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n29zp\" (UniqueName: \"kubernetes.io/projected/0452390d-e7b4-49f2-9501-a7db296d6f29-kube-api-access-n29zp\") pod \"test-curl-8894-394926\" (UID: \"0452390d-e7b4-49f2-9501-a7db296d6f29\") " pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:15.099132 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:15.099103    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29zp\" (UniqueName: \"kubernetes.io/projected/0452390d-e7b4-49f2-9501-a7db296d6f29-kube-api-access-n29zp\") pod \"test-curl-8894-394926\" (UID: \"0452390d-e7b4-49f2-9501-a7db296d6f29\") " pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:15.120476 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:15.120418    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8894-2d9a54" container="test-curl-8894-2d9a54"
Sep 18 18:20:15.283035 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:15.283007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:15.565073 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:15.565021    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-394926"]
Sep 18 18:20:15.566146 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:15.566111    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0452390d_e7b4_49f2_9501_a7db296d6f29.slice/crio-91d3803a38699b6e9b299e0f8393215f115ddf160bb25c70c44cd37081298a97 WatchSource:0}: Error finding container 91d3803a38699b6e9b299e0f8393215f115ddf160bb25c70c44cd37081298a97: Status 404 returned error can't find the container with id 91d3803a38699b6e9b299e0f8393215f115ddf160bb25c70c44cd37081298a97
Sep 18 18:20:16.129126 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.129098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-394926" event={"ID":"0452390d-e7b4-49f2-9501-a7db296d6f29","Type":"ContainerStarted","Data":"626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9"}
Sep 18 18:20:16.129500 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.129144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-394926" event={"ID":"0452390d-e7b4-49f2-9501-a7db296d6f29","Type":"ContainerStarted","Data":"91d3803a38699b6e9b299e0f8393215f115ddf160bb25c70c44cd37081298a97"}
Sep 18 18:20:16.129500 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.129357    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8894-2d9a54" podUID="f6a795d5-cc96-4b53-8ae7-740cc712d5a4" containerName="test-curl-8894-2d9a54" containerID="cri-o://1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41" gracePeriod=2
Sep 18 18:20:16.144680 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.144638    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8894-394926" podStartSLOduration=2.1446238 podStartE2EDuration="2.1446238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:16.14265421 +0000 UTC m=+1199.569866967" watchObservedRunningTime="2026-09-18 18:20:16.1446238 +0000 UTC m=+1199.571836554"
Sep 18 18:20:16.295471 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.295447    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:16.404893 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.404793    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svps8\" (UniqueName: \"kubernetes.io/projected/181dd568-2673-442f-8f8f-bf84c9e1b969-kube-api-access-svps8\") pod \"181dd568-2673-442f-8f8f-bf84c9e1b969\" (UID: \"181dd568-2673-442f-8f8f-bf84c9e1b969\") "
Sep 18 18:20:16.407000 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.406976    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181dd568-2673-442f-8f8f-bf84c9e1b969-kube-api-access-svps8" (OuterVolumeSpecName: "kube-api-access-svps8") pod "181dd568-2673-442f-8f8f-bf84c9e1b969" (UID: "181dd568-2673-442f-8f8f-bf84c9e1b969"). InnerVolumeSpecName "kube-api-access-svps8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:16.506472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.506436    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svps8\" (UniqueName: \"kubernetes.io/projected/181dd568-2673-442f-8f8f-bf84c9e1b969-kube-api-access-svps8\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.733523 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.733435    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8894-394926"]
Sep 18 18:20:16.736863 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.736834    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8894-394926"]
Sep 18 18:20:16.850226 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.850194    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8894-ff5748"]
Sep 18 18:20:16.859286 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.859251    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0452390d-e7b4-49f2-9501-a7db296d6f29" containerName="test-curl-8894-394926"
Sep 18 18:20:16.859544 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.859508    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0452390d-e7b4-49f2-9501-a7db296d6f29" containerName="test-curl-8894-394926"
Sep 18 18:20:16.860018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.859963    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0452390d-e7b4-49f2-9501-a7db296d6f29" containerName="test-curl-8894-394926"
Sep 18 18:20:16.864715 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.864681    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-ff5748"]
Sep 18 18:20:16.864834 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.864765    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:16.911180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.911155    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv82w\" (UniqueName: \"kubernetes.io/projected/995b5f8e-59ac-4ded-9050-7ff1a7a37866-kube-api-access-vv82w\") pod \"test-curl-8894-ff5748\" (UID: \"995b5f8e-59ac-4ded-9050-7ff1a7a37866\") " pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:16.921222 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:16.921192    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="181dd568-2673-442f-8f8f-bf84c9e1b969" path="/var/lib/kubelet/pods/181dd568-2673-442f-8f8f-bf84c9e1b969/volumes"
Sep 18 18:20:17.011795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.011720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv82w\" (UniqueName: \"kubernetes.io/projected/995b5f8e-59ac-4ded-9050-7ff1a7a37866-kube-api-access-vv82w\") pod \"test-curl-8894-ff5748\" (UID: \"995b5f8e-59ac-4ded-9050-7ff1a7a37866\") " pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:17.019771 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.019745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv82w\" (UniqueName: \"kubernetes.io/projected/995b5f8e-59ac-4ded-9050-7ff1a7a37866-kube-api-access-vv82w\") pod \"test-curl-8894-ff5748\" (UID: \"995b5f8e-59ac-4ded-9050-7ff1a7a37866\") " pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:17.137686 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.137646    2582 generic.go:356] "Generic (PLEG): container finished" podID="181dd568-2673-442f-8f8f-bf84c9e1b969" containerID="372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873" exitCode=137
Sep 18 18:20:17.138168 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.137704    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-e7d1ab"
Sep 18 18:20:17.138168 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.137749    2582 scope.go:117] "RemoveContainer" containerID="372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873"
Sep 18 18:20:17.138168 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:17.137893    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8894-394926" container="test-curl-8894-394926"
Sep 18 18:20:17.149693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.149671    2582 scope.go:117] "RemoveContainer" containerID="372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873"
Sep 18 18:20:17.149966 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:17.149948    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873\": container with ID starting with 372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873 not found: ID does not exist" containerID="372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873"
Sep 18 18:20:17.150017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.149975    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873"} err="failed to get container status \"372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873\": rpc error: code = NotFound desc = could not find container \"372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873\": container with ID starting with 372330a3f9e54391a32e5d8f18fe9580cfef758007971bf71dbccc2e258eb873 not found: ID does not exist"
Sep 18 18:20:17.175685 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.175662    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:17.447601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:17.447577    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8894-ff5748"]
Sep 18 18:20:17.449779 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:17.449753    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod995b5f8e_59ac_4ded_9050_7ff1a7a37866.slice/crio-c796c6f7d8b501dc25cc0cd9be3dc0c7f9af58de807752b74fb4d2fc94243cad WatchSource:0}: Error finding container c796c6f7d8b501dc25cc0cd9be3dc0c7f9af58de807752b74fb4d2fc94243cad: Status 404 returned error can't find the container with id c796c6f7d8b501dc25cc0cd9be3dc0c7f9af58de807752b74fb4d2fc94243cad
Sep 18 18:20:18.147768 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.147737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-ff5748" event={"ID":"995b5f8e-59ac-4ded-9050-7ff1a7a37866","Type":"ContainerStarted","Data":"3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04"}
Sep 18 18:20:18.148160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.147782    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8894-ff5748" event={"ID":"995b5f8e-59ac-4ded-9050-7ff1a7a37866","Type":"ContainerStarted","Data":"c796c6f7d8b501dc25cc0cd9be3dc0c7f9af58de807752b74fb4d2fc94243cad"}
Sep 18 18:20:18.148160 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.147900    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8894-394926" podUID="0452390d-e7b4-49f2-9501-a7db296d6f29" containerName="test-curl-8894-394926" containerID="cri-o://626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9" gracePeriod=2
Sep 18 18:20:18.164157 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.164110    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8894-ff5748" podStartSLOduration=2.164096197 podStartE2EDuration="2.164096197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:18.161487603 +0000 UTC m=+1201.588700357" watchObservedRunningTime="2026-09-18 18:20:18.164096197 +0000 UTC m=+1201.591308951"
Sep 18 18:20:18.327373 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.327347    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:18.425382 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.425300    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcdt5\" (UniqueName: \"kubernetes.io/projected/f6a795d5-cc96-4b53-8ae7-740cc712d5a4-kube-api-access-wcdt5\") pod \"f6a795d5-cc96-4b53-8ae7-740cc712d5a4\" (UID: \"f6a795d5-cc96-4b53-8ae7-740cc712d5a4\") "
Sep 18 18:20:18.427418 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.427393    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6a795d5-cc96-4b53-8ae7-740cc712d5a4-kube-api-access-wcdt5" (OuterVolumeSpecName: "kube-api-access-wcdt5") pod "f6a795d5-cc96-4b53-8ae7-740cc712d5a4" (UID: "f6a795d5-cc96-4b53-8ae7-740cc712d5a4"). InnerVolumeSpecName "kube-api-access-wcdt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:18.527121 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.527094    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcdt5\" (UniqueName: \"kubernetes.io/projected/f6a795d5-cc96-4b53-8ae7-740cc712d5a4-kube-api-access-wcdt5\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:20:18.631393 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.631358    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"]
Sep 18 18:20:18.635675 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.635654    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.638213 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.638190    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-011541-serving-cert\""
Sep 18 18:20:18.644775 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.644749    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"]
Sep 18 18:20:18.729571 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.729458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.729571 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.729521    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-946g9\" (UniqueName: \"kubernetes.io/projected/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-kube-api-access-946g9\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.729817 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.729661    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.729817 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.729774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.732825 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.732781    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8894-ff5748"]
Sep 18 18:20:18.737471 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.737447    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8894-ff5748"]
Sep 18 18:20:18.830845 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.830814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.831012 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.830855    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-946g9\" (UniqueName: \"kubernetes.io/projected/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-kube-api-access-946g9\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.831012 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.830924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.831012 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.830958    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.831570 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.831550    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.833139 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.833120    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-011541-serving-cert\""
Sep 18 18:20:18.833456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.833437    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.838448 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.838414    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-946g9\" (UniqueName: \"kubernetes.io/projected/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-kube-api-access-946g9\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:18.841180 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:18.841160    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-serving-cert: secret "maas-api-e2e-shared-a-w2-011541-serving-cert" not found
Sep 18 18:20:18.841309 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:18.841224    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls podName:65732db1-93bc-4dfb-9fdc-b0e3770b93ba nodeName:}" failed. No retries permitted until 2026-09-18 18:20:19.341206831 +0000 UTC m=+1202.768419564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls") pod "maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" (UID: "65732db1-93bc-4dfb-9fdc-b0e3770b93ba") : secret "maas-api-e2e-shared-a-w2-011541-serving-cert" not found
Sep 18 18:20:18.932793 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.932765    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6a795d5-cc96-4b53-8ae7-740cc712d5a4" path="/var/lib/kubelet/pods/f6a795d5-cc96-4b53-8ae7-740cc712d5a4/volumes"
Sep 18 18:20:18.939353 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.939327    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:20:18.941127 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:18.941104    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:20:19.159444 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.159396    2582 generic.go:356] "Generic (PLEG): container finished" podID="f6a795d5-cc96-4b53-8ae7-740cc712d5a4" containerID="1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41" exitCode=137
Sep 18 18:20:19.159444 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.159447    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-2d9a54"
Sep 18 18:20:19.159966 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.159502    2582 scope.go:117] "RemoveContainer" containerID="1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41"
Sep 18 18:20:19.159966 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:19.159742    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8894-ff5748" container="test-curl-8894-ff5748"
Sep 18 18:20:19.169324 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.169304    2582 scope.go:117] "RemoveContainer" containerID="1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41"
Sep 18 18:20:19.169602 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:19.169582    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41\": container with ID starting with 1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41 not found: ID does not exist" containerID="1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41"
Sep 18 18:20:19.169678 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.169611    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41"} err="failed to get container status \"1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41\": rpc error: code = NotFound desc = could not find container \"1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41\": container with ID starting with 1db9b81667bd6fc245015a0255d220d709aa21507514217cc2da5ecc4aa64a41 not found: ID does not exist"
Sep 18 18:20:19.437224 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.437187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:19.439568 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.439541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:19.552992 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.552954    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:19.859988 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:19.859953    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"]
Sep 18 18:20:19.862112 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:19.862078    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65732db1_93bc_4dfb_9fdc_b0e3770b93ba.slice/crio-02a78323a75aa956c1cb1a0632d63635eb552ccfc6613a76ba2aa7875cd7bd29 WatchSource:0}: Error finding container 02a78323a75aa956c1cb1a0632d63635eb552ccfc6613a76ba2aa7875cd7bd29: Status 404 returned error can't find the container with id 02a78323a75aa956c1cb1a0632d63635eb552ccfc6613a76ba2aa7875cd7bd29
Sep 18 18:20:20.168633 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.168608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" event={"ID":"65732db1-93bc-4dfb-9fdc-b0e3770b93ba","Type":"ContainerStarted","Data":"02a78323a75aa956c1cb1a0632d63635eb552ccfc6613a76ba2aa7875cd7bd29"}
Sep 18 18:20:20.168935 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.168699    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8894-ff5748" podUID="995b5f8e-59ac-4ded-9050-7ff1a7a37866" containerName="test-curl-8894-ff5748" containerID="cri-o://3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04" gracePeriod=2
Sep 18 18:20:20.337378 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.337355    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:20.447289 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.447267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n29zp\" (UniqueName: \"kubernetes.io/projected/0452390d-e7b4-49f2-9501-a7db296d6f29-kube-api-access-n29zp\") pod \"0452390d-e7b4-49f2-9501-a7db296d6f29\" (UID: \"0452390d-e7b4-49f2-9501-a7db296d6f29\") "
Sep 18 18:20:20.449724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.449691    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0452390d-e7b4-49f2-9501-a7db296d6f29-kube-api-access-n29zp" (OuterVolumeSpecName: "kube-api-access-n29zp") pod "0452390d-e7b4-49f2-9501-a7db296d6f29" (UID: "0452390d-e7b4-49f2-9501-a7db296d6f29"). InnerVolumeSpecName "kube-api-access-n29zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:20.549101 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.549066    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n29zp\" (UniqueName: \"kubernetes.io/projected/0452390d-e7b4-49f2-9501-a7db296d6f29-kube-api-access-n29zp\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:20:20.921204 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:20.921172    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0452390d-e7b4-49f2-9501-a7db296d6f29" path="/var/lib/kubelet/pods/0452390d-e7b4-49f2-9501-a7db296d6f29/volumes"
Sep 18 18:20:21.176821 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.176731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" event={"ID":"65732db1-93bc-4dfb-9fdc-b0e3770b93ba","Type":"ContainerStarted","Data":"0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e"}
Sep 18 18:20:21.177287 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.176835    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:21.178230 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.178209    2582 generic.go:356] "Generic (PLEG): container finished" podID="0452390d-e7b4-49f2-9501-a7db296d6f29" containerID="626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9" exitCode=137
Sep 18 18:20:21.178338 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.178246    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-394926"
Sep 18 18:20:21.178338 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.178307    2582 scope.go:117] "RemoveContainer" containerID="626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9"
Sep 18 18:20:21.190723 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.190646    2582 scope.go:117] "RemoveContainer" containerID="626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9"
Sep 18 18:20:21.191229 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:21.191210    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9\": container with ID starting with 626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9 not found: ID does not exist" containerID="626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9"
Sep 18 18:20:21.191303 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.191241    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9"} err="failed to get container status \"626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9\": rpc error: code = NotFound desc = could not find container \"626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9\": container with ID starting with 626ed60649325ecec32c04627503f0543e25e3175fd66c886fa04217165c31b9 not found: ID does not exist"
Sep 18 18:20:21.200363 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:21.200323    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" podStartSLOduration=2.870183219 podStartE2EDuration="3.200310293s" totalImagesPullingTime="330.127074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:21.197838706 +0000 UTC m=+1204.625051460" watchObservedRunningTime="2026-09-18 18:20:21.200310293 +0000 UTC m=+1204.627523047"
Sep 18 18:20:22.345612 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:22.345588    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:22.469784 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:22.469703    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv82w\" (UniqueName: \"kubernetes.io/projected/995b5f8e-59ac-4ded-9050-7ff1a7a37866-kube-api-access-vv82w\") pod \"995b5f8e-59ac-4ded-9050-7ff1a7a37866\" (UID: \"995b5f8e-59ac-4ded-9050-7ff1a7a37866\") "
Sep 18 18:20:22.471854 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:22.471830    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/995b5f8e-59ac-4ded-9050-7ff1a7a37866-kube-api-access-vv82w" (OuterVolumeSpecName: "kube-api-access-vv82w") pod "995b5f8e-59ac-4ded-9050-7ff1a7a37866" (UID: "995b5f8e-59ac-4ded-9050-7ff1a7a37866"). InnerVolumeSpecName "kube-api-access-vv82w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:22.570856 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:22.570817    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv82w\" (UniqueName: \"kubernetes.io/projected/995b5f8e-59ac-4ded-9050-7ff1a7a37866-kube-api-access-vv82w\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:20:22.921565 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:22.921529    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="995b5f8e-59ac-4ded-9050-7ff1a7a37866" path="/var/lib/kubelet/pods/995b5f8e-59ac-4ded-9050-7ff1a7a37866/volumes"
Sep 18 18:20:23.036732 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:23.036698    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:20:23.199980 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:23.199877    2582 generic.go:356] "Generic (PLEG): container finished" podID="995b5f8e-59ac-4ded-9050-7ff1a7a37866" containerID="3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04" exitCode=137
Sep 18 18:20:23.199980 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:23.199929    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8894-ff5748"
Sep 18 18:20:23.200239 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:23.199991    2582 scope.go:117] "RemoveContainer" containerID="3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04"
Sep 18 18:20:23.209982 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:23.209823    2582 scope.go:117] "RemoveContainer" containerID="3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04"
Sep 18 18:20:23.210153 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:23.210131    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04\": container with ID starting with 3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04 not found: ID does not exist" containerID="3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04"
Sep 18 18:20:23.210258 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:23.210158    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04"} err="failed to get container status \"3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04\": rpc error: code = NotFound desc = could not find container \"3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04\": container with ID starting with 3738e7899c791c7ebe3164f559c445bc265e0c3f6b94b4818e577be0522aba04 not found: ID does not exist"
Sep 18 18:20:27.087986 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.087942    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"]
Sep 18 18:20:27.088734 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.088708    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="995b5f8e-59ac-4ded-9050-7ff1a7a37866" containerName="test-curl-8894-ff5748"
Sep 18 18:20:27.088866 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.088752    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="995b5f8e-59ac-4ded-9050-7ff1a7a37866" containerName="test-curl-8894-ff5748"
Sep 18 18:20:27.088938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.088915    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="995b5f8e-59ac-4ded-9050-7ff1a7a37866" containerName="test-curl-8894-ff5748"
Sep 18 18:20:27.093714 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.093692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.095958 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.095936    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6fa1a2a2-serving-cert\""
Sep 18 18:20:27.107396 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.107370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"]
Sep 18 18:20:27.191932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.191899    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:20:27.217131 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.217074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-metrics-certs\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.217311 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.217207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.217311 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.217301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413f994e-fca5-4b1f-bd0b-b6aec8d95060-openshift-service-ca\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.217389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.217345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llfph\" (UniqueName: \"kubernetes.io/projected/413f994e-fca5-4b1f-bd0b-b6aec8d95060-kube-api-access-llfph\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.318111 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.318077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-metrics-certs\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.318308 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.318152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.318308 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.318220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413f994e-fca5-4b1f-bd0b-b6aec8d95060-openshift-service-ca\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.318431 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:27.318304    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-serving-cert: secret "maas-api-e2e-mt-6fa1a2a2-serving-cert" not found
Sep 18 18:20:27.318431 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.318366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llfph\" (UniqueName: \"kubernetes.io/projected/413f994e-fca5-4b1f-bd0b-b6aec8d95060-kube-api-access-llfph\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.318431 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:27.318396    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls podName:413f994e-fca5-4b1f-bd0b-b6aec8d95060 nodeName:}" failed. No retries permitted until 2026-09-18 18:20:27.818374294 +0000 UTC m=+1211.245587043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls") pod "maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" (UID: "413f994e-fca5-4b1f-bd0b-b6aec8d95060") : secret "maas-api-e2e-mt-6fa1a2a2-serving-cert" not found
Sep 18 18:20:27.318917 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.318901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413f994e-fca5-4b1f-bd0b-b6aec8d95060-openshift-service-ca\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.320675 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.320650    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-metrics-certs\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.328928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.328901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llfph\" (UniqueName: \"kubernetes.io/projected/413f994e-fca5-4b1f-bd0b-b6aec8d95060-kube-api-access-llfph\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.825266 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.825231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:27.827796 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:27.827771    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls\") pod \"maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:28.007215 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:28.007176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:28.291080 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:28.291034    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"]
Sep 18 18:20:28.292397 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:28.292369    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413f994e_fca5_4b1f_bd0b_b6aec8d95060.slice/crio-77e4027c4d8278ad62728bc489bb4e0cf3b51c5ef16a470dfba09f54467b2f37 WatchSource:0}: Error finding container 77e4027c4d8278ad62728bc489bb4e0cf3b51c5ef16a470dfba09f54467b2f37: Status 404 returned error can't find the container with id 77e4027c4d8278ad62728bc489bb4e0cf3b51c5ef16a470dfba09f54467b2f37
Sep 18 18:20:29.241815 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:29.241779    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" event={"ID":"413f994e-fca5-4b1f-bd0b-b6aec8d95060","Type":"ContainerStarted","Data":"f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704"}
Sep 18 18:20:29.241989 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:29.241840    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" event={"ID":"413f994e-fca5-4b1f-bd0b-b6aec8d95060","Type":"ContainerStarted","Data":"77e4027c4d8278ad62728bc489bb4e0cf3b51c5ef16a470dfba09f54467b2f37"}
Sep 18 18:20:29.242032 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:29.241981    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:29.259271 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:29.259215    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" podStartSLOduration=1.880020015 podStartE2EDuration="2.259200719s" totalImagesPullingTime="379.180704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:29.25821855 +0000 UTC m=+1212.685431318" watchObservedRunningTime="2026-09-18 18:20:29.259200719 +0000 UTC m=+1212.686413473"
Sep 18 18:20:33.095083 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:33.095030    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:20:33.610069 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:33.610015    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:20:35.253580 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:35.253547    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:20:43.150710 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:43.150675    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:20:48.657277 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:48.657235    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:20:50.203587 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.203513    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"]
Sep 18 18:20:50.209387 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.209360    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.211678 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.211656    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-99602c-serving-cert\""
Sep 18 18:20:50.218304 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.218275    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"]
Sep 18 18:20:50.241687 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.241656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.241801 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.241706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.241801 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.241780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9nmt\" (UniqueName: \"kubernetes.io/projected/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-kube-api-access-w9nmt\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.241801 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.241800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.342753 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.342718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.342928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.342776    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.342928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.342859    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9nmt\" (UniqueName: \"kubernetes.io/projected/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-kube-api-access-w9nmt\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.342928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.342888    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.343086 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:50.343016    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-serving-cert: secret "maas-api-e2e-shared-b-w2-99602c-serving-cert" not found
Sep 18 18:20:50.343130 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:50.343120    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls podName:929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c nodeName:}" failed. No retries permitted until 2026-09-18 18:20:50.843098837 +0000 UTC m=+1234.270311571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls") pod "maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" (UID: "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c") : secret "maas-api-e2e-shared-b-w2-99602c-serving-cert" not found
Sep 18 18:20:50.343535 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.343512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.345493 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.345471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.351018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.351000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9nmt\" (UniqueName: \"kubernetes.io/projected/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-kube-api-access-w9nmt\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.848944 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.848910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:50.851528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:50.851492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:51.123301 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:51.123204    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:51.403612 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:51.403578    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"]
Sep 18 18:20:51.404159 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:20:51.404133    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929563e7_7caf_4dd7_bb6b_4ca7f3a62a1c.slice/crio-c176d61b5061d9845fb998008812726f6e39fb26c1454b25eb9406be38a6ca66 WatchSource:0}: Error finding container c176d61b5061d9845fb998008812726f6e39fb26c1454b25eb9406be38a6ca66: Status 404 returned error can't find the container with id c176d61b5061d9845fb998008812726f6e39fb26c1454b25eb9406be38a6ca66
Sep 18 18:20:52.415487 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:52.415451    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" event={"ID":"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c","Type":"ContainerStarted","Data":"6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e"}
Sep 18 18:20:52.415862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:52.415504    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" event={"ID":"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c","Type":"ContainerStarted","Data":"c176d61b5061d9845fb998008812726f6e39fb26c1454b25eb9406be38a6ca66"}
Sep 18 18:20:52.415862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:52.415550    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:20:52.434217 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:52.434165    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" podStartSLOduration=2.016039958 podStartE2EDuration="2.434150119s" totalImagesPullingTime="418.110161ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:52.431846085 +0000 UTC m=+1235.859058852" watchObservedRunningTime="2026-09-18 18:20:52.434150119 +0000 UTC m=+1235.861362873"
Sep 18 18:20:53.163010 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:20:53.162976    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:20:58.428327 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:20:58.428294    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:21:03.220388 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:21:03.220355    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:21:03.619460 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:21:03.619426    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:21:13.278755 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:21:13.278719    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:21:14.509159 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:21:14.509127    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:21:14.509604 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:21:14.509124    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:21:18.645941 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:21:18.645908    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6a795d5_cc96_4b53_8ae7_740cc712d5a4.slice/crio-bea24af14dfd4959190a5aa60f11bb7d80b9df2139b6d59ea3854d7ae170c7ce\": RecentStats: unable to find data in memory cache]"
Sep 18 18:21:19.007576 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:21:19.007499    2582 scope.go:117] "RemoveContainer" containerID="167e7c036d69dcaa9b8ea8bdd07570157ceeef5123ca440429c40cf6b1191e58"
Sep 18 18:21:19.017266 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:21:19.017248    2582 scope.go:117] "RemoveContainer" containerID="00de11bb3e062834c837845d728eccc0a2f438dc9cc6dda172ae9d3944dd6c53"
Sep 18 18:21:47.970472 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:21:47.970408    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"]
Sep 18 18:21:47.973437 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:21:47.973415    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-039dbc-29829255-k7blt"]
Sep 18 18:21:48.923342 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:21:48.923304    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3f83e7-e3a1-4b82-825e-2a348aa7c801" path="/var/lib/kubelet/pods/fc3f83e7-e3a1-4b82-825e-2a348aa7c801/volumes"
Sep 18 18:22:19.059906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:19.059873    2582 scope.go:117] "RemoveContainer" containerID="2527339ad97d7f5b49bc039762abd1c3a4b70d7b29cb7d56e5aa03c6ac55d6a5"
Sep 18 18:22:19.069396 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:19.069374    2582 scope.go:117] "RemoveContainer" containerID="7e80632c4a64df192ca53d0ff05717e9ad1d412b4ff0e71a226e1e44b0d53b5a"
Sep 18 18:22:39.274368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.274333    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"]
Sep 18 18:22:39.274843 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.274635    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" podUID="413f994e-fca5-4b1f-bd0b-b6aec8d95060" containerName="maas-api" containerID="cri-o://f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704" gracePeriod=30
Sep 18 18:22:39.543998 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.543975    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:22:39.614223 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.614188    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls\") pod \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") "
Sep 18 18:22:39.614421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.614249    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413f994e-fca5-4b1f-bd0b-b6aec8d95060-openshift-service-ca\") pod \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") "
Sep 18 18:22:39.614421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.614311    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llfph\" (UniqueName: \"kubernetes.io/projected/413f994e-fca5-4b1f-bd0b-b6aec8d95060-kube-api-access-llfph\") pod \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") "
Sep 18 18:22:39.614421 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.614379    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-metrics-certs\") pod \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\" (UID: \"413f994e-fca5-4b1f-bd0b-b6aec8d95060\") "
Sep 18 18:22:39.614750 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.614704    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413f994e-fca5-4b1f-bd0b-b6aec8d95060-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "413f994e-fca5-4b1f-bd0b-b6aec8d95060" (UID: "413f994e-fca5-4b1f-bd0b-b6aec8d95060"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:22:39.617272 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.617224    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "413f994e-fca5-4b1f-bd0b-b6aec8d95060" (UID: "413f994e-fca5-4b1f-bd0b-b6aec8d95060"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:22:39.618238 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.618203    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "413f994e-fca5-4b1f-bd0b-b6aec8d95060" (UID: "413f994e-fca5-4b1f-bd0b-b6aec8d95060"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:22:39.618653 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.618622    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413f994e-fca5-4b1f-bd0b-b6aec8d95060-kube-api-access-llfph" (OuterVolumeSpecName: "kube-api-access-llfph") pod "413f994e-fca5-4b1f-bd0b-b6aec8d95060" (UID: "413f994e-fca5-4b1f-bd0b-b6aec8d95060"). InnerVolumeSpecName "kube-api-access-llfph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:22:39.715652 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.715608    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:22:39.715652 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.715645    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413f994e-fca5-4b1f-bd0b-b6aec8d95060-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:22:39.715652 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.715659    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llfph\" (UniqueName: \"kubernetes.io/projected/413f994e-fca5-4b1f-bd0b-b6aec8d95060-kube-api-access-llfph\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:22:39.715882 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:39.715671    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/413f994e-fca5-4b1f-bd0b-b6aec8d95060-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:22:40.118601 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.118567    2582 generic.go:356] "Generic (PLEG): container finished" podID="413f994e-fca5-4b1f-bd0b-b6aec8d95060" containerID="f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704" exitCode=0
Sep 18 18:22:40.118776 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.118628    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"
Sep 18 18:22:40.118776 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.118651    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" event={"ID":"413f994e-fca5-4b1f-bd0b-b6aec8d95060","Type":"ContainerDied","Data":"f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704"}
Sep 18 18:22:40.118776 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.118713    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl" event={"ID":"413f994e-fca5-4b1f-bd0b-b6aec8d95060","Type":"ContainerDied","Data":"77e4027c4d8278ad62728bc489bb4e0cf3b51c5ef16a470dfba09f54467b2f37"}
Sep 18 18:22:40.118776 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.118741    2582 scope.go:117] "RemoveContainer" containerID="f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704"
Sep 18 18:22:40.130041 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.130018    2582 scope.go:117] "RemoveContainer" containerID="f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704"
Sep 18 18:22:40.130438 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:22:40.130415    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704\": container with ID starting with f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704 not found: ID does not exist" containerID="f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704"
Sep 18 18:22:40.130506 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.130447    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704"} err="failed to get container status \"f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704\": rpc error: code = NotFound desc = could not find container \"f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704\": container with ID starting with f977e42899a0698ecd888ad816669382f545672890f62d0e8c2bb441370c5704 not found: ID does not exist"
Sep 18 18:22:40.142088 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.142040    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"]
Sep 18 18:22:40.146247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.146225    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6fa1a2a2-967bc9697-7lxhl"]
Sep 18 18:22:40.923023 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:22:40.922992    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="413f994e-fca5-4b1f-bd0b-b6aec8d95060" path="/var/lib/kubelet/pods/413f994e-fca5-4b1f-bd0b-b6aec8d95060/volumes"
Sep 18 18:23:19.095746 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:23:19.095711    2582 scope.go:117] "RemoveContainer" containerID="cb1e99ad2e1f661ca39edc80fd2e89a4569aaa49c9b9911f244312b7d52c66f5"
Sep 18 18:24:14.516760 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.516666    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"]
Sep 18 18:24:14.521971 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.521939    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"]
Sep 18 18:24:14.522227 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.522198    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb" podUID="0efd2b7a-6791-471b-818e-5de06768e5da" containerName="payload-pre-processing" containerID="cri-o://25529e1013580e5ac11c0745461120c222d450fc932410c7b795daf96cbba113" gracePeriod=30
Sep 18 18:24:14.723972 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.723935    2582 generic.go:356] "Generic (PLEG): container finished" podID="0efd2b7a-6791-471b-818e-5de06768e5da" containerID="25529e1013580e5ac11c0745461120c222d450fc932410c7b795daf96cbba113" exitCode=0
Sep 18 18:24:14.724165 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.724006    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb" event={"ID":"0efd2b7a-6791-471b-818e-5de06768e5da","Type":"ContainerDied","Data":"25529e1013580e5ac11c0745461120c222d450fc932410c7b795daf96cbba113"}
Sep 18 18:24:14.724376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.724350    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml" podUID="8aee3bb0-5d28-45f3-a374-c507854901f3" containerName="payload-processing" containerID="cri-o://f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670" gracePeriod=30
Sep 18 18:24:14.878870 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.878842    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:24:14.950707 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.950678    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg6vn\" (UniqueName: \"kubernetes.io/projected/0efd2b7a-6791-471b-818e-5de06768e5da-kube-api-access-wg6vn\") pod \"0efd2b7a-6791-471b-818e-5de06768e5da\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") "
Sep 18 18:24:14.950855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.950767    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0efd2b7a-6791-471b-818e-5de06768e5da-plugins-config-volume\") pod \"0efd2b7a-6791-471b-818e-5de06768e5da\" (UID: \"0efd2b7a-6791-471b-818e-5de06768e5da\") "
Sep 18 18:24:14.951276 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.951210    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0efd2b7a-6791-471b-818e-5de06768e5da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0efd2b7a-6791-471b-818e-5de06768e5da" (UID: "0efd2b7a-6791-471b-818e-5de06768e5da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:24:14.954393 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:14.954353    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efd2b7a-6791-471b-818e-5de06768e5da-kube-api-access-wg6vn" (OuterVolumeSpecName: "kube-api-access-wg6vn") pod "0efd2b7a-6791-471b-818e-5de06768e5da" (UID: "0efd2b7a-6791-471b-818e-5de06768e5da"). InnerVolumeSpecName "kube-api-access-wg6vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:15.011631 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.011609    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:24:15.051762 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.051722    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/8aee3bb0-5d28-45f3-a374-c507854901f3-kube-api-access-n57fd\") pod \"8aee3bb0-5d28-45f3-a374-c507854901f3\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") "
Sep 18 18:24:15.051935 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.051851    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8aee3bb0-5d28-45f3-a374-c507854901f3-plugins-config-volume\") pod \"8aee3bb0-5d28-45f3-a374-c507854901f3\" (UID: \"8aee3bb0-5d28-45f3-a374-c507854901f3\") "
Sep 18 18:24:15.052231 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.052205    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aee3bb0-5d28-45f3-a374-c507854901f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8aee3bb0-5d28-45f3-a374-c507854901f3" (UID: "8aee3bb0-5d28-45f3-a374-c507854901f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:24:15.052340 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.052268    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg6vn\" (UniqueName: \"kubernetes.io/projected/0efd2b7a-6791-471b-818e-5de06768e5da-kube-api-access-wg6vn\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:15.052340 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.052298    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0efd2b7a-6791-471b-818e-5de06768e5da-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:15.054461 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.054432    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aee3bb0-5d28-45f3-a374-c507854901f3-kube-api-access-n57fd" (OuterVolumeSpecName: "kube-api-access-n57fd") pod "8aee3bb0-5d28-45f3-a374-c507854901f3" (UID: "8aee3bb0-5d28-45f3-a374-c507854901f3"). InnerVolumeSpecName "kube-api-access-n57fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:15.153190 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.153109    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8aee3bb0-5d28-45f3-a374-c507854901f3-plugins-config-volume\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:15.153190 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.153146    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/8aee3bb0-5d28-45f3-a374-c507854901f3-kube-api-access-n57fd\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:15.731854 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.731819    2582 generic.go:356] "Generic (PLEG): container finished" podID="8aee3bb0-5d28-45f3-a374-c507854901f3" containerID="f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670" exitCode=0
Sep 18 18:24:15.732310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.731862    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"
Sep 18 18:24:15.732310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.731907    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml" event={"ID":"8aee3bb0-5d28-45f3-a374-c507854901f3","Type":"ContainerDied","Data":"f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670"}
Sep 18 18:24:15.732310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.731968    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml" event={"ID":"8aee3bb0-5d28-45f3-a374-c507854901f3","Type":"ContainerDied","Data":"880a44353876173611269ae3ab18a9379644ab6718727151d2d8b74da1541ebe"}
Sep 18 18:24:15.732310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.731999    2582 scope.go:117] "RemoveContainer" containerID="f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670"
Sep 18 18:24:15.733442 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.733404    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"
Sep 18 18:24:15.733510 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.733406    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb" event={"ID":"0efd2b7a-6791-471b-818e-5de06768e5da","Type":"ContainerDied","Data":"90cf6b190c6ac7fb3e3f586c23b1c34c3e5f6a18bf1d5276738c6b0438035944"}
Sep 18 18:24:15.746153 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.746112    2582 scope.go:117] "RemoveContainer" containerID="f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670"
Sep 18 18:24:15.746658 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:24:15.746634    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670\": container with ID starting with f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670 not found: ID does not exist" containerID="f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670"
Sep 18 18:24:15.746770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.746664    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670"} err="failed to get container status \"f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670\": rpc error: code = NotFound desc = could not find container \"f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670\": container with ID starting with f250f477732f2fe85ba676b641d9882b6b78a1f826a6d7eb4bd18be765cb1670 not found: ID does not exist"
Sep 18 18:24:15.746770 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.746681    2582 scope.go:117] "RemoveContainer" containerID="25529e1013580e5ac11c0745461120c222d450fc932410c7b795daf96cbba113"
Sep 18 18:24:15.758144 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.758114    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"]
Sep 18 18:24:15.761790 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.761768    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-37c153-59b57ff795-thwml"]
Sep 18 18:24:15.772693 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.772671    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"]
Sep 18 18:24:15.776358 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:15.776338    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-37c153-559788cfd-7mvbb"]
Sep 18 18:24:16.921623 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:16.921591    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0efd2b7a-6791-471b-818e-5de06768e5da" path="/var/lib/kubelet/pods/0efd2b7a-6791-471b-818e-5de06768e5da/volumes"
Sep 18 18:24:16.921995 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:16.921961    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aee3bb0-5d28-45f3-a374-c507854901f3" path="/var/lib/kubelet/pods/8aee3bb0-5d28-45f3-a374-c507854901f3/volumes"
Sep 18 18:24:21.289605 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:21.289572    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"]
Sep 18 18:24:21.290140 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:21.289834    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy" containerID="cri-o://901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6" gracePeriod=30
Sep 18 18:24:26.547379 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.547354    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:24:26.667789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.667702    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-token\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.667789 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.667751    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istiod-ca-cert\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668013 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.667795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcvf6\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-kube-api-access-jcvf6\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668013 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.667846    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-socket\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668013 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.667886    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-certs\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668013 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.667955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-credential-socket\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668013 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668000    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-envoy\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668175    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:26.668318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668242    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:24:26.668318 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668258    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:26.668477 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668434    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-podinfo\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668537 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668490    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:26.668537 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668498    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-data\") pod \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\" (UID: \"48f32b99-d00c-4f0b-a6a8-35d71b659bd4\") "
Sep 18 18:24:26.668749 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.668723    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-data" (OuterVolumeSpecName: "istio-data") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:26.669186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.669116    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-data\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.669186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.669142    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istiod-ca-cert\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.669186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.669158    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-socket\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.669186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.669171    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-workload-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.669186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.669183    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-credential-socket\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.670216 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.670180    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-token" (OuterVolumeSpecName: "istio-token") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:26.670326 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.670264    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:26.670326 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.670301    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-kube-api-access-jcvf6" (OuterVolumeSpecName: "kube-api-access-jcvf6") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "kube-api-access-jcvf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:26.670598 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.670583    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "48f32b99-d00c-4f0b-a6a8-35d71b659bd4" (UID: "48f32b99-d00c-4f0b-a6a8-35d71b659bd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:24:26.770215 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.770187    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcvf6\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-kube-api-access-jcvf6\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.770215 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.770210    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-envoy\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.770215 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.770218    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-podinfo\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.770396 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.770227    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48f32b99-d00c-4f0b-a6a8-35d71b659bd4-istio-token\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:24:26.807708 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.807672    2582 generic.go:356] "Generic (PLEG): container finished" podID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerID="901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6" exitCode=0
Sep 18 18:24:26.807828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.807776    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"
Sep 18 18:24:26.807828 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.807785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" event={"ID":"48f32b99-d00c-4f0b-a6a8-35d71b659bd4","Type":"ContainerDied","Data":"901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6"}
Sep 18 18:24:26.807904 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.807833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7" event={"ID":"48f32b99-d00c-4f0b-a6a8-35d71b659bd4","Type":"ContainerDied","Data":"8ac043449074b92df96adf8b9e50676830bd200d507e8a04407a69e5eaaf6b1a"}
Sep 18 18:24:26.807904 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.807853    2582 scope.go:117] "RemoveContainer" containerID="901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6"
Sep 18 18:24:26.818447 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.818432    2582 scope.go:117] "RemoveContainer" containerID="b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"
Sep 18 18:24:26.826802 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.826787    2582 scope.go:117] "RemoveContainer" containerID="901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6"
Sep 18 18:24:26.827071 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:24:26.827035    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6\": container with ID starting with 901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6 not found: ID does not exist" containerID="901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6"
Sep 18 18:24:26.827118 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.827081    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6"} err="failed to get container status \"901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6\": rpc error: code = NotFound desc = could not find container \"901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6\": container with ID starting with 901d4262cde85aa4e8481eeab41771264ad0d2de711b6268bb6381ba03ee6fe6 not found: ID does not exist"
Sep 18 18:24:26.827118 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.827097    2582 scope.go:117] "RemoveContainer" containerID="b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"
Sep 18 18:24:26.827340 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:24:26.827321    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c\": container with ID starting with b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c not found: ID does not exist" containerID="b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"
Sep 18 18:24:26.827386 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.827348    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c"} err="failed to get container status \"b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c\": rpc error: code = NotFound desc = could not find container \"b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c\": container with ID starting with b0fbbc1d79bc6fe885193e9f2f137b51df6f831f71ba5a878ad1b44e8112e92c not found: ID does not exist"
Sep 18 18:24:26.834091 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.834070    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"]
Sep 18 18:24:26.841747 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.841729    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-37c153-openshift-default-67c588ff5b-qcns7"]
Sep 18 18:24:26.920570 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:24:26.920504    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" path="/var/lib/kubelet/pods/48f32b99-d00c-4f0b-a6a8-35d71b659bd4/volumes"
Sep 18 18:25:11.789745 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.789709    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"]
Sep 18 18:25:11.790777 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790753    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413f994e-fca5-4b1f-bd0b-b6aec8d95060" containerName="maas-api"
Sep 18 18:25:11.790777 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790778    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="413f994e-fca5-4b1f-bd0b-b6aec8d95060" containerName="maas-api"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790798    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790804    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790810    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aee3bb0-5d28-45f3-a374-c507854901f3" containerName="payload-processing"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790815    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aee3bb0-5d28-45f3-a374-c507854901f3" containerName="payload-processing"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790826    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790831    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790837    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0efd2b7a-6791-471b-818e-5de06768e5da" containerName="payload-pre-processing"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790843    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0efd2b7a-6791-471b-818e-5de06768e5da" containerName="payload-pre-processing"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790906    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790914    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="48f32b99-d00c-4f0b-a6a8-35d71b659bd4" containerName="istio-proxy"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790921    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="413f994e-fca5-4b1f-bd0b-b6aec8d95060" containerName="maas-api"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790929    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0efd2b7a-6791-471b-818e-5de06768e5da" containerName="payload-pre-processing"
Sep 18 18:25:11.790932 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.790936    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aee3bb0-5d28-45f3-a374-c507854901f3" containerName="payload-processing"
Sep 18 18:25:11.794359 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.794339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:11.797089 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.797064    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-29f0cb-serving-cert\""
Sep 18 18:25:11.804043 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.804014    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"]
Sep 18 18:25:11.910501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.910467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176bb1ec-1038-4005-975e-a008e8d298ba-openshift-service-ca\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:11.910501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.910505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-metrics-certs\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:11.910710 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.910611    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr6wp\" (UniqueName: \"kubernetes.io/projected/176bb1ec-1038-4005-975e-a008e8d298ba-kube-api-access-zr6wp\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:11.910710 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:11.910631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.011899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.011858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176bb1ec-1038-4005-975e-a008e8d298ba-openshift-service-ca\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.011899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.011901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-metrics-certs\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.012171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.012018    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr6wp\" (UniqueName: \"kubernetes.io/projected/176bb1ec-1038-4005-975e-a008e8d298ba-kube-api-access-zr6wp\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.012171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.012067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.012257 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:25:12.012176    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-serving-cert: secret "maas-api-e2e-api-b-29f0cb-serving-cert" not found
Sep 18 18:25:12.012312 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:25:12.012259    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls podName:176bb1ec-1038-4005-975e-a008e8d298ba nodeName:}" failed. No retries permitted until 2026-09-18 18:25:12.51223638 +0000 UTC m=+1495.939449115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls") pod "maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" (UID: "176bb1ec-1038-4005-975e-a008e8d298ba") : secret "maas-api-e2e-api-b-29f0cb-serving-cert" not found
Sep 18 18:25:12.012557 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.012538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176bb1ec-1038-4005-975e-a008e8d298ba-openshift-service-ca\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.014851 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.014824    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-metrics-certs\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.022434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.022410    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr6wp\" (UniqueName: \"kubernetes.io/projected/176bb1ec-1038-4005-975e-a008e8d298ba-kube-api-access-zr6wp\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.516394 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.516354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.518855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.518829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls\") pod \"maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.707524 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.707486    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:12.985622 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.985595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"]
Sep 18 18:25:12.986821 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:25:12.986786    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176bb1ec_1038_4005_975e_a008e8d298ba.slice/crio-f4cc6048bab5fae6a808845d307c64b1d57fecc271e4fefcc6a074cd90dd5ab7 WatchSource:0}: Error finding container f4cc6048bab5fae6a808845d307c64b1d57fecc271e4fefcc6a074cd90dd5ab7: Status 404 returned error can't find the container with id f4cc6048bab5fae6a808845d307c64b1d57fecc271e4fefcc6a074cd90dd5ab7
Sep 18 18:25:12.988226 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:12.988208    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:25:13.113247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:13.113210    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" event={"ID":"176bb1ec-1038-4005-975e-a008e8d298ba","Type":"ContainerStarted","Data":"f4cc6048bab5fae6a808845d307c64b1d57fecc271e4fefcc6a074cd90dd5ab7"}
Sep 18 18:25:14.121281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.121246    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" event={"ID":"176bb1ec-1038-4005-975e-a008e8d298ba","Type":"ContainerStarted","Data":"5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e"}
Sep 18 18:25:14.121742 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.121341    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:14.138712 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.138661    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" podStartSLOduration=2.758230713 podStartE2EDuration="3.138648597s" totalImagesPullingTime="380.417884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:25:14.136834796 +0000 UTC m=+1497.564047552" watchObservedRunningTime="2026-09-18 18:25:14.138648597 +0000 UTC m=+1497.565861383"
Sep 18 18:25:14.364199 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.364167    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8897-55fc52"]
Sep 18 18:25:14.367938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.367919    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:14.370793 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.370758    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxj55\""
Sep 18 18:25:14.374420 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.374367    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8897-55fc52"]
Sep 18 18:25:14.434497 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.434468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p5cd\" (UniqueName: \"kubernetes.io/projected/1552153b-f9c1-4f8a-9b95-93537eea98fd-kube-api-access-5p5cd\") pod \"test-curl-8897-55fc52\" (UID: \"1552153b-f9c1-4f8a-9b95-93537eea98fd\") " pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:14.535332 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.535306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p5cd\" (UniqueName: \"kubernetes.io/projected/1552153b-f9c1-4f8a-9b95-93537eea98fd-kube-api-access-5p5cd\") pod \"test-curl-8897-55fc52\" (UID: \"1552153b-f9c1-4f8a-9b95-93537eea98fd\") " pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:14.543187 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.543170    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p5cd\" (UniqueName: \"kubernetes.io/projected/1552153b-f9c1-4f8a-9b95-93537eea98fd-kube-api-access-5p5cd\") pod \"test-curl-8897-55fc52\" (UID: \"1552153b-f9c1-4f8a-9b95-93537eea98fd\") " pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:14.679638 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.679573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:14.950782 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:14.950702    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8897-55fc52"]
Sep 18 18:25:14.954306 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:25:14.954270    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1552153b_f9c1_4f8a_9b95_93537eea98fd.slice/crio-338f88648bd3f6c19333ea838b0f7844562be6e7b04c778b965a422c0fdb5315 WatchSource:0}: Error finding container 338f88648bd3f6c19333ea838b0f7844562be6e7b04c778b965a422c0fdb5315: Status 404 returned error can't find the container with id 338f88648bd3f6c19333ea838b0f7844562be6e7b04c778b965a422c0fdb5315
Sep 18 18:25:15.129102 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:15.129064    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8897-55fc52" event={"ID":"1552153b-f9c1-4f8a-9b95-93537eea98fd","Type":"ContainerStarted","Data":"55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4"}
Sep 18 18:25:15.129503 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:15.129113    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8897-55fc52" event={"ID":"1552153b-f9c1-4f8a-9b95-93537eea98fd","Type":"ContainerStarted","Data":"338f88648bd3f6c19333ea838b0f7844562be6e7b04c778b965a422c0fdb5315"}
Sep 18 18:25:15.145300 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:15.145242    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8897-55fc52" podStartSLOduration=1.14522819 podStartE2EDuration="1.14522819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:25:15.142222252 +0000 UTC m=+1498.569435009" watchObservedRunningTime="2026-09-18 18:25:15.14522819 +0000 UTC m=+1498.572440944"
Sep 18 18:25:15.758603 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:15.758523    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8897-55fc52"]
Sep 18 18:25:15.766031 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:15.765994    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8897-55fc52"]
Sep 18 18:25:16.135597 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:25:16.135568    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8897-55fc52" container="test-curl-8897-55fc52"
Sep 18 18:25:17.144827 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:17.144151    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8897-55fc52" podUID="1552153b-f9c1-4f8a-9b95-93537eea98fd" containerName="test-curl-8897-55fc52" containerID="cri-o://55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4" gracePeriod=2
Sep 18 18:25:18.983916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:18.983889    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:25:18.987180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:18.987160    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:25:19.301457 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:19.301427    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:19.387247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:19.387212    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p5cd\" (UniqueName: \"kubernetes.io/projected/1552153b-f9c1-4f8a-9b95-93537eea98fd-kube-api-access-5p5cd\") pod \"1552153b-f9c1-4f8a-9b95-93537eea98fd\" (UID: \"1552153b-f9c1-4f8a-9b95-93537eea98fd\") "
Sep 18 18:25:19.389660 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:19.389625    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1552153b-f9c1-4f8a-9b95-93537eea98fd-kube-api-access-5p5cd" (OuterVolumeSpecName: "kube-api-access-5p5cd") pod "1552153b-f9c1-4f8a-9b95-93537eea98fd" (UID: "1552153b-f9c1-4f8a-9b95-93537eea98fd"). InnerVolumeSpecName "kube-api-access-5p5cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:25:19.488020 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:19.487990    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p5cd\" (UniqueName: \"kubernetes.io/projected/1552153b-f9c1-4f8a-9b95-93537eea98fd-kube-api-access-5p5cd\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:25:20.135202 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.135172    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:25:20.174721 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.174680    2582 generic.go:356] "Generic (PLEG): container finished" podID="1552153b-f9c1-4f8a-9b95-93537eea98fd" containerID="55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4" exitCode=137
Sep 18 18:25:20.174905 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.174723    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:20.174905 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.174782    2582 scope.go:117] "RemoveContainer" containerID="55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4"
Sep 18 18:25:20.185762 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.185741    2582 scope.go:117] "RemoveContainer" containerID="55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4"
Sep 18 18:25:20.186116 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:25:20.186086    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4\": container with ID starting with 55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4 not found: ID does not exist" containerID="55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4"
Sep 18 18:25:20.186248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.186124    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4"} err="failed to get container status \"55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4\": rpc error: code = NotFound desc = could not find container \"55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4\": container with ID starting with 55d9e25b15f76c97004f699a72b81e5b4328d0a05ad49f39cdc36e9f2b5729a4 not found: ID does not exist"
Sep 18 18:25:20.186633 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:25:20.186599    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8897-55fc52\" is forbidden: User \"system:node:ip-10-0-135-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-209.ec2.internal' and this object" podUID="1552153b-f9c1-4f8a-9b95-93537eea98fd" pod="opendatahub/test-curl-8897-55fc52"
Sep 18 18:25:20.920690 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:25:20.920655    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1552153b-f9c1-4f8a-9b95-93537eea98fd" path="/var/lib/kubelet/pods/1552153b-f9c1-4f8a-9b95-93537eea98fd/volumes"
Sep 18 18:26:19.164007 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:19.163975    2582 scope.go:117] "RemoveContainer" containerID="0812d172f39421bf3355fde7e4c07ac2bd62684d10efa81adc30858809d048c0"
Sep 18 18:26:20.277927 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.277890    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"]
Sep 18 18:26:20.278370 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.278165    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" podUID="176bb1ec-1038-4005-975e-a008e8d298ba" containerName="maas-api" containerID="cri-o://5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e" gracePeriod=30
Sep 18 18:26:20.536306 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.536283    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:26:20.572189 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.572156    2582 generic.go:356] "Generic (PLEG): container finished" podID="176bb1ec-1038-4005-975e-a008e8d298ba" containerID="5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e" exitCode=0
Sep 18 18:26:20.572378 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.572202    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"
Sep 18 18:26:20.572378 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.572237    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" event={"ID":"176bb1ec-1038-4005-975e-a008e8d298ba","Type":"ContainerDied","Data":"5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e"}
Sep 18 18:26:20.572378 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.572300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8" event={"ID":"176bb1ec-1038-4005-975e-a008e8d298ba","Type":"ContainerDied","Data":"f4cc6048bab5fae6a808845d307c64b1d57fecc271e4fefcc6a074cd90dd5ab7"}
Sep 18 18:26:20.572378 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.572328    2582 scope.go:117] "RemoveContainer" containerID="5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e"
Sep 18 18:26:20.582134 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.582114    2582 scope.go:117] "RemoveContainer" containerID="5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e"
Sep 18 18:26:20.582470 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:26:20.582451    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e\": container with ID starting with 5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e not found: ID does not exist" containerID="5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e"
Sep 18 18:26:20.582566 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.582479    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e"} err="failed to get container status \"5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e\": rpc error: code = NotFound desc = could not find container \"5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e\": container with ID starting with 5dfcb3f2e38c19be3bd6a566b15d15ac17bdd303dde935875ee96274e60b903e not found: ID does not exist"
Sep 18 18:26:20.588817 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.588791    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176bb1ec-1038-4005-975e-a008e8d298ba-openshift-service-ca\") pod \"176bb1ec-1038-4005-975e-a008e8d298ba\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") "
Sep 18 18:26:20.588926 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.588825    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr6wp\" (UniqueName: \"kubernetes.io/projected/176bb1ec-1038-4005-975e-a008e8d298ba-kube-api-access-zr6wp\") pod \"176bb1ec-1038-4005-975e-a008e8d298ba\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") "
Sep 18 18:26:20.588996 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.588939    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-metrics-certs\") pod \"176bb1ec-1038-4005-975e-a008e8d298ba\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") "
Sep 18 18:26:20.588996 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.588986    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls\") pod \"176bb1ec-1038-4005-975e-a008e8d298ba\" (UID: \"176bb1ec-1038-4005-975e-a008e8d298ba\") "
Sep 18 18:26:20.589164 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.589138    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/176bb1ec-1038-4005-975e-a008e8d298ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "176bb1ec-1038-4005-975e-a008e8d298ba" (UID: "176bb1ec-1038-4005-975e-a008e8d298ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:20.589428 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.589411    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176bb1ec-1038-4005-975e-a008e8d298ba-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:20.591340 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.591306    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "176bb1ec-1038-4005-975e-a008e8d298ba" (UID: "176bb1ec-1038-4005-975e-a008e8d298ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:20.591634 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.591614    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "176bb1ec-1038-4005-975e-a008e8d298ba" (UID: "176bb1ec-1038-4005-975e-a008e8d298ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:20.591706 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.591651    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/176bb1ec-1038-4005-975e-a008e8d298ba-kube-api-access-zr6wp" (OuterVolumeSpecName: "kube-api-access-zr6wp") pod "176bb1ec-1038-4005-975e-a008e8d298ba" (UID: "176bb1ec-1038-4005-975e-a008e8d298ba"). InnerVolumeSpecName "kube-api-access-zr6wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:20.690772 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.690721    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:20.690772 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.690766    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176bb1ec-1038-4005-975e-a008e8d298ba-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:20.690990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.690782    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr6wp\" (UniqueName: \"kubernetes.io/projected/176bb1ec-1038-4005-975e-a008e8d298ba-kube-api-access-zr6wp\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:20.898009 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.897974    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"]
Sep 18 18:26:20.904501 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.904476    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-29f0cb-6bf6d97569-2khr8"]
Sep 18 18:26:20.921367 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:20.921335    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="176bb1ec-1038-4005-975e-a008e8d298ba" path="/var/lib/kubelet/pods/176bb1ec-1038-4005-975e-a008e8d298ba/volumes"
Sep 18 18:26:38.200485 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.200448    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"]
Sep 18 18:26:38.201336 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.201302    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" podUID="fad27a2d-eefe-4b43-b0ae-1cde44277917" containerName="maas-api" containerID="cri-o://49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2" gracePeriod=30
Sep 18 18:26:38.460409 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.460336    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:26:38.572983 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.572950    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad27a2d-eefe-4b43-b0ae-1cde44277917-openshift-service-ca\") pod \"fad27a2d-eefe-4b43-b0ae-1cde44277917\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") "
Sep 18 18:26:38.573181 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.573042    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v85t6\" (UniqueName: \"kubernetes.io/projected/fad27a2d-eefe-4b43-b0ae-1cde44277917-kube-api-access-v85t6\") pod \"fad27a2d-eefe-4b43-b0ae-1cde44277917\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") "
Sep 18 18:26:38.573181 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.573140    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls\") pod \"fad27a2d-eefe-4b43-b0ae-1cde44277917\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") "
Sep 18 18:26:38.573181 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.573167    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-metrics-certs\") pod \"fad27a2d-eefe-4b43-b0ae-1cde44277917\" (UID: \"fad27a2d-eefe-4b43-b0ae-1cde44277917\") "
Sep 18 18:26:38.573376 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.573347    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad27a2d-eefe-4b43-b0ae-1cde44277917-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fad27a2d-eefe-4b43-b0ae-1cde44277917" (UID: "fad27a2d-eefe-4b43-b0ae-1cde44277917"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:38.573611 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.573581    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad27a2d-eefe-4b43-b0ae-1cde44277917-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:38.575347 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.575320    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad27a2d-eefe-4b43-b0ae-1cde44277917-kube-api-access-v85t6" (OuterVolumeSpecName: "kube-api-access-v85t6") pod "fad27a2d-eefe-4b43-b0ae-1cde44277917" (UID: "fad27a2d-eefe-4b43-b0ae-1cde44277917"). InnerVolumeSpecName "kube-api-access-v85t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:38.575447 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.575352    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fad27a2d-eefe-4b43-b0ae-1cde44277917" (UID: "fad27a2d-eefe-4b43-b0ae-1cde44277917"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:38.575488 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.575448    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fad27a2d-eefe-4b43-b0ae-1cde44277917" (UID: "fad27a2d-eefe-4b43-b0ae-1cde44277917"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:38.675230 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.675198    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:38.675230 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.675229    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fad27a2d-eefe-4b43-b0ae-1cde44277917-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:38.675436 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.675241    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v85t6\" (UniqueName: \"kubernetes.io/projected/fad27a2d-eefe-4b43-b0ae-1cde44277917-kube-api-access-v85t6\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:38.688175 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.688141    2582 generic.go:356] "Generic (PLEG): container finished" podID="fad27a2d-eefe-4b43-b0ae-1cde44277917" containerID="49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2" exitCode=0
Sep 18 18:26:38.688341 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.688191    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"
Sep 18 18:26:38.688341 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.688214    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" event={"ID":"fad27a2d-eefe-4b43-b0ae-1cde44277917","Type":"ContainerDied","Data":"49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2"}
Sep 18 18:26:38.688341 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.688265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz" event={"ID":"fad27a2d-eefe-4b43-b0ae-1cde44277917","Type":"ContainerDied","Data":"54dd46feb19c0494ac2037e226df33f4b06356aa6bddca6b1ff2744d75d2001d"}
Sep 18 18:26:38.688341 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.688292    2582 scope.go:117] "RemoveContainer" containerID="49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2"
Sep 18 18:26:38.698780 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.698763    2582 scope.go:117] "RemoveContainer" containerID="49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2"
Sep 18 18:26:38.699081 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:26:38.699036    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2\": container with ID starting with 49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2 not found: ID does not exist" containerID="49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2"
Sep 18 18:26:38.699144 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.699090    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2"} err="failed to get container status \"49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2\": rpc error: code = NotFound desc = could not find container \"49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2\": container with ID starting with 49fe6d4e48999e494c62783c5d0fcbb0f2baf7b05e467e1ca84c8b62d56c8de2 not found: ID does not exist"
Sep 18 18:26:38.711214 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.711157    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"]
Sep 18 18:26:38.714746 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.714723    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f5e6d4-669c8747d4-7mcjz"]
Sep 18 18:26:38.921449 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:38.921418    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad27a2d-eefe-4b43-b0ae-1cde44277917" path="/var/lib/kubelet/pods/fad27a2d-eefe-4b43-b0ae-1cde44277917/volumes"
Sep 18 18:26:55.715434 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:55.715398    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"]
Sep 18 18:26:55.716366 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:55.715658    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5" podUID="611a5f86-321e-4ab3-a32c-95237850af4c" containerName="maas-api" containerID="cri-o://256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92" gracePeriod=30
Sep 18 18:26:55.966586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:55.966526    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:26:56.044938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.044901    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-metrics-certs\") pod \"611a5f86-321e-4ab3-a32c-95237850af4c\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") "
Sep 18 18:26:56.045180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.045009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611a5f86-321e-4ab3-a32c-95237850af4c-openshift-service-ca\") pod \"611a5f86-321e-4ab3-a32c-95237850af4c\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") "
Sep 18 18:26:56.045180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.045066    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crqfp\" (UniqueName: \"kubernetes.io/projected/611a5f86-321e-4ab3-a32c-95237850af4c-kube-api-access-crqfp\") pod \"611a5f86-321e-4ab3-a32c-95237850af4c\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") "
Sep 18 18:26:56.045180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.045099    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-maas-api-tls\") pod \"611a5f86-321e-4ab3-a32c-95237850af4c\" (UID: \"611a5f86-321e-4ab3-a32c-95237850af4c\") "
Sep 18 18:26:56.045355 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.045331    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611a5f86-321e-4ab3-a32c-95237850af4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "611a5f86-321e-4ab3-a32c-95237850af4c" (UID: "611a5f86-321e-4ab3-a32c-95237850af4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:56.045547 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.045523    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611a5f86-321e-4ab3-a32c-95237850af4c-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.047523 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.047487    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611a5f86-321e-4ab3-a32c-95237850af4c-kube-api-access-crqfp" (OuterVolumeSpecName: "kube-api-access-crqfp") pod "611a5f86-321e-4ab3-a32c-95237850af4c" (UID: "611a5f86-321e-4ab3-a32c-95237850af4c"). InnerVolumeSpecName "kube-api-access-crqfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:56.047609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.047522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "611a5f86-321e-4ab3-a32c-95237850af4c" (UID: "611a5f86-321e-4ab3-a32c-95237850af4c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:56.047609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.047539    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "611a5f86-321e-4ab3-a32c-95237850af4c" (UID: "611a5f86-321e-4ab3-a32c-95237850af4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:56.146817 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.146774    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crqfp\" (UniqueName: \"kubernetes.io/projected/611a5f86-321e-4ab3-a32c-95237850af4c-kube-api-access-crqfp\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.146817 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.146816    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.147019 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.146830    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611a5f86-321e-4ab3-a32c-95237850af4c-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.804457 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.804425    2582 generic.go:356] "Generic (PLEG): container finished" podID="611a5f86-321e-4ab3-a32c-95237850af4c" containerID="256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92" exitCode=0
Sep 18 18:26:56.804864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.804468    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"
Sep 18 18:26:56.804864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.804505    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5" event={"ID":"611a5f86-321e-4ab3-a32c-95237850af4c","Type":"ContainerDied","Data":"256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92"}
Sep 18 18:26:56.804864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.804566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5" event={"ID":"611a5f86-321e-4ab3-a32c-95237850af4c","Type":"ContainerDied","Data":"0dba83d413cceada63355b20aba6f21650aa08056429777ecbb51dac4d10e657"}
Sep 18 18:26:56.804864 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.804593    2582 scope.go:117] "RemoveContainer" containerID="256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92"
Sep 18 18:26:56.817867 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.817849    2582 scope.go:117] "RemoveContainer" containerID="256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92"
Sep 18 18:26:56.818212 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:26:56.818194    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92\": container with ID starting with 256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92 not found: ID does not exist" containerID="256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92"
Sep 18 18:26:56.818288 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.818221    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92"} err="failed to get container status \"256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92\": rpc error: code = NotFound desc = could not find container \"256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92\": container with ID starting with 256d5fedebbc52af86038b58e005b7795f340bd2fd893f42cf348752ff02ee92 not found: ID does not exist"
Sep 18 18:26:56.829572 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.829550    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"]
Sep 18 18:26:56.832144 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.832123    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1f7000-c977b7449-fcsc5"]
Sep 18 18:26:56.920493 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:26:56.920458    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611a5f86-321e-4ab3-a32c-95237850af4c" path="/var/lib/kubelet/pods/611a5f86-321e-4ab3-a32c-95237850af4c/volumes"
Sep 18 18:28:30.084137 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084103    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"]
Sep 18 18:28:30.084796 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084779    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1552153b-f9c1-4f8a-9b95-93537eea98fd" containerName="test-curl-8897-55fc52"
Sep 18 18:28:30.084847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084799    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1552153b-f9c1-4f8a-9b95-93537eea98fd" containerName="test-curl-8897-55fc52"
Sep 18 18:28:30.084847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084814    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad27a2d-eefe-4b43-b0ae-1cde44277917" containerName="maas-api"
Sep 18 18:28:30.084847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084825    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad27a2d-eefe-4b43-b0ae-1cde44277917" containerName="maas-api"
Sep 18 18:28:30.084847 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084838    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="176bb1ec-1038-4005-975e-a008e8d298ba" containerName="maas-api"
Sep 18 18:28:30.084973 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084848    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="176bb1ec-1038-4005-975e-a008e8d298ba" containerName="maas-api"
Sep 18 18:28:30.084973 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084861    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611a5f86-321e-4ab3-a32c-95237850af4c" containerName="maas-api"
Sep 18 18:28:30.084973 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084870    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="611a5f86-321e-4ab3-a32c-95237850af4c" containerName="maas-api"
Sep 18 18:28:30.085080 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084975    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="611a5f86-321e-4ab3-a32c-95237850af4c" containerName="maas-api"
Sep 18 18:28:30.085080 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084987    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1552153b-f9c1-4f8a-9b95-93537eea98fd" containerName="test-curl-8897-55fc52"
Sep 18 18:28:30.085080 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.084998    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad27a2d-eefe-4b43-b0ae-1cde44277917" containerName="maas-api"
Sep 18 18:28:30.085080 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.085013    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="176bb1ec-1038-4005-975e-a008e8d298ba" containerName="maas-api"
Sep 18 18:28:30.088722 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.088701    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.091017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.090994    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-46c63c-serving-cert\""
Sep 18 18:28:30.097527 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.097407    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"]
Sep 18 18:28:30.225171 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.225140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbs6r\" (UniqueName: \"kubernetes.io/projected/77a2c67f-f184-4828-944f-114a39586056-kube-api-access-zbs6r\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.225346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.225195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.225346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.225213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77a2c67f-f184-4828-944f-114a39586056-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.225346 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.225231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-metrics-certs\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.326368 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.326330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbs6r\" (UniqueName: \"kubernetes.io/projected/77a2c67f-f184-4828-944f-114a39586056-kube-api-access-zbs6r\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.326528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.326385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.326528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.326415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77a2c67f-f184-4828-944f-114a39586056-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.326528 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.326437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-metrics-certs\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.326640 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:28:30.326541    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-serving-cert: secret "maas-api-e2e-ipp-b-46c63c-serving-cert" not found
Sep 18 18:28:30.326640 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:28:30.326609    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls podName:77a2c67f-f184-4828-944f-114a39586056 nodeName:}" failed. No retries permitted until 2026-09-18 18:28:30.826592418 +0000 UTC m=+1694.253805150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls") pod "maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" (UID: "77a2c67f-f184-4828-944f-114a39586056") : secret "maas-api-e2e-ipp-b-46c63c-serving-cert" not found
Sep 18 18:28:30.327061 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.327022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77a2c67f-f184-4828-944f-114a39586056-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.328972 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.328955    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-metrics-certs\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.334365 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.334312    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbs6r\" (UniqueName: \"kubernetes.io/projected/77a2c67f-f184-4828-944f-114a39586056-kube-api-access-zbs6r\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.831795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.831760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:30.834405 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:30.834379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls\") pod \"maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:31.001668 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:31.001619    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:31.305098 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:31.302983    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"]
Sep 18 18:28:31.377683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:31.377644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" event={"ID":"77a2c67f-f184-4828-944f-114a39586056","Type":"ContainerStarted","Data":"29e01b0099a81621afce45b072799b78bda164fbbbbbfd6e61a4750b61ae467a"}
Sep 18 18:28:32.384629 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:32.384594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" event={"ID":"77a2c67f-f184-4828-944f-114a39586056","Type":"ContainerStarted","Data":"efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584"}
Sep 18 18:28:32.385034 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:32.384715    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:32.407744 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:32.407702    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" podStartSLOduration=2.076290038 podStartE2EDuration="2.407688433s" totalImagesPullingTime="331.398395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:28:32.405333499 +0000 UTC m=+1695.832546254" watchObservedRunningTime="2026-09-18 18:28:32.407688433 +0000 UTC m=+1695.834901186"
Sep 18 18:28:38.396295 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:38.396254    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:28:57.108934 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:57.108907    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:28:57.223895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:28:57.223871    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:29:28.538224 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:28.538194    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:29:28.657783 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:28.657753    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:29:35.645711 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.645673    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"]
Sep 18 18:29:35.649857 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.649834    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.652297 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.652275    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-12b560-serving-cert\""
Sep 18 18:29:35.658915 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.658891    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"]
Sep 18 18:29:35.726299 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.726264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec66c98-8eed-45cd-87ed-be4cce7a973f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.726443 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.726308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.726443 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.726329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fzx8\" (UniqueName: \"kubernetes.io/projected/bec66c98-8eed-45cd-87ed-be4cce7a973f-kube-api-access-8fzx8\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.726443 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.726428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.827866 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.827830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec66c98-8eed-45cd-87ed-be4cce7a973f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.828075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.827878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.828075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.827897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fzx8\" (UniqueName: \"kubernetes.io/projected/bec66c98-8eed-45cd-87ed-be4cce7a973f-kube-api-access-8fzx8\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.828075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.827959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.828545 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.828520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec66c98-8eed-45cd-87ed-be4cce7a973f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.830606 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.830583    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.830692 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.830614    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.835615 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.835596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fzx8\" (UniqueName: \"kubernetes.io/projected/bec66c98-8eed-45cd-87ed-be4cce7a973f-kube-api-access-8fzx8\") pod \"maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:35.962261 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:35.962189    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:36.262617 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:36.262580    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"]
Sep 18 18:29:36.263760 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:29:36.263728    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec66c98_8eed_45cd_87ed_be4cce7a973f.slice/crio-6d1d7b34662a5425e7664f3d433e70e905876a1fbc57bc4dbf5dba42440e1a82 WatchSource:0}: Error finding container 6d1d7b34662a5425e7664f3d433e70e905876a1fbc57bc4dbf5dba42440e1a82: Status 404 returned error can't find the container with id 6d1d7b34662a5425e7664f3d433e70e905876a1fbc57bc4dbf5dba42440e1a82
Sep 18 18:29:36.776821 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:36.776713    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr" event={"ID":"bec66c98-8eed-45cd-87ed-be4cce7a973f","Type":"ContainerStarted","Data":"32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb"}
Sep 18 18:29:36.776821 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:36.776772    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr" event={"ID":"bec66c98-8eed-45cd-87ed-be4cce7a973f","Type":"ContainerStarted","Data":"6d1d7b34662a5425e7664f3d433e70e905876a1fbc57bc4dbf5dba42440e1a82"}
Sep 18 18:29:36.777287 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:36.776910    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:29:36.794558 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:36.794508    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr" podStartSLOduration=1.422397251 podStartE2EDuration="1.794493139s" totalImagesPullingTime="372.095888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:29:36.791551173 +0000 UTC m=+1760.218763931" watchObservedRunningTime="2026-09-18 18:29:36.794493139 +0000 UTC m=+1760.221705893"
Sep 18 18:29:42.788616 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:29:42.788587    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:30:00.151375 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.151285    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"]
Sep 18 18:30:00.156013 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.155986    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"
Sep 18 18:30:00.158994 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.158969    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:30:00.163896 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.163870    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"]
Sep 18 18:30:00.170302 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.170286    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"
Sep 18 18:30:00.453629 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.453591    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"]
Sep 18 18:30:00.455138 ip-10-0-135-209 kubenswrapper[2582]: W0918 18:30:00.455107    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff793fd_bac4_4d2f_9bd2_5e6c631a8aba.slice/crio-eb06660e898e0247b5291a13a33ab4ad2d5c89c668c02a1d0b64312cdd663965 WatchSource:0}: Error finding container eb06660e898e0247b5291a13a33ab4ad2d5c89c668c02a1d0b64312cdd663965: Status 404 returned error can't find the container with id eb06660e898e0247b5291a13a33ab4ad2d5c89c668c02a1d0b64312cdd663965
Sep 18 18:30:00.931426 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.931385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9" event={"ID":"8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba","Type":"ContainerStarted","Data":"a6697ad097a5348e07c46b0b82e4218a40bf4b4977e99939e34c22f736242a7d"}
Sep 18 18:30:00.931598 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.931444    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9" event={"ID":"8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba","Type":"ContainerStarted","Data":"eb06660e898e0247b5291a13a33ab4ad2d5c89c668c02a1d0b64312cdd663965"}
Sep 18 18:30:00.947360 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:00.947315    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9" podStartSLOduration=0.947301231 podStartE2EDuration="947.301231ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:30:00.945105993 +0000 UTC m=+1784.372318747" watchObservedRunningTime="2026-09-18 18:30:00.947301231 +0000 UTC m=+1784.374514007"
Sep 18 18:30:01.938935 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:01.938900    2582 generic.go:356] "Generic (PLEG): container finished" podID="8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba" containerID="a6697ad097a5348e07c46b0b82e4218a40bf4b4977e99939e34c22f736242a7d" exitCode=0
Sep 18 18:30:01.939357 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:01.938954    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9" event={"ID":"8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba","Type":"ContainerDied","Data":"a6697ad097a5348e07c46b0b82e4218a40bf4b4977e99939e34c22f736242a7d"}
Sep 18 18:30:03.085122 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:03.085097    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"
Sep 18 18:30:03.952385 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:03.952354    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"
Sep 18 18:30:03.952556 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:03.952401    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9" event={"ID":"8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba","Type":"ContainerDied","Data":"eb06660e898e0247b5291a13a33ab4ad2d5c89c668c02a1d0b64312cdd663965"}
Sep 18 18:30:03.952556 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:03.952459    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb06660e898e0247b5291a13a33ab4ad2d5c89c668c02a1d0b64312cdd663965"
Sep 18 18:30:04.663683 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.663647    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"]
Sep 18 18:30:04.664176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.663912    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr" podUID="bec66c98-8eed-45cd-87ed-be4cce7a973f" containerName="maas-api" containerID="cri-o://32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb" gracePeriod=30
Sep 18 18:30:04.922018 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.921960    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:30:04.961087 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.961029    2582 generic.go:356] "Generic (PLEG): container finished" podID="bec66c98-8eed-45cd-87ed-be4cce7a973f" containerID="32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb" exitCode=0
Sep 18 18:30:04.961232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.961112    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr" event={"ID":"bec66c98-8eed-45cd-87ed-be4cce7a973f","Type":"ContainerDied","Data":"32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb"}
Sep 18 18:30:04.961232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.961163    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr" event={"ID":"bec66c98-8eed-45cd-87ed-be4cce7a973f","Type":"ContainerDied","Data":"6d1d7b34662a5425e7664f3d433e70e905876a1fbc57bc4dbf5dba42440e1a82"}
Sep 18 18:30:04.961232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.961184    2582 scope.go:117] "RemoveContainer" containerID="32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb"
Sep 18 18:30:04.961232 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.961118    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"
Sep 18 18:30:04.973457 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.973437    2582 scope.go:117] "RemoveContainer" containerID="32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb"
Sep 18 18:30:04.973807 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:30:04.973786    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb\": container with ID starting with 32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb not found: ID does not exist" containerID="32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb"
Sep 18 18:30:04.973895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:04.973814    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb"} err="failed to get container status \"32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb\": rpc error: code = NotFound desc = could not find container \"32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb\": container with ID starting with 32c68d4be91df0895695c1d7828e3c57a896e0173ea6185aeec8aeb79b2e03eb not found: ID does not exist"
Sep 18 18:30:05.019223 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.019194    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-metrics-certs\") pod \"bec66c98-8eed-45cd-87ed-be4cce7a973f\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") "
Sep 18 18:30:05.019413 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.019269    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec66c98-8eed-45cd-87ed-be4cce7a973f-openshift-service-ca\") pod \"bec66c98-8eed-45cd-87ed-be4cce7a973f\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") "
Sep 18 18:30:05.019413 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.019288    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-maas-api-tls\") pod \"bec66c98-8eed-45cd-87ed-be4cce7a973f\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") "
Sep 18 18:30:05.019413 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.019348    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fzx8\" (UniqueName: \"kubernetes.io/projected/bec66c98-8eed-45cd-87ed-be4cce7a973f-kube-api-access-8fzx8\") pod \"bec66c98-8eed-45cd-87ed-be4cce7a973f\" (UID: \"bec66c98-8eed-45cd-87ed-be4cce7a973f\") "
Sep 18 18:30:05.019747 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.019716    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec66c98-8eed-45cd-87ed-be4cce7a973f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bec66c98-8eed-45cd-87ed-be4cce7a973f" (UID: "bec66c98-8eed-45cd-87ed-be4cce7a973f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:05.020179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.020156    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec66c98-8eed-45cd-87ed-be4cce7a973f-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:30:05.022098 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.022065    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bec66c98-8eed-45cd-87ed-be4cce7a973f" (UID: "bec66c98-8eed-45cd-87ed-be4cce7a973f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:30:05.022212 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.022158    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bec66c98-8eed-45cd-87ed-be4cce7a973f" (UID: "bec66c98-8eed-45cd-87ed-be4cce7a973f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:30:05.022212 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.022180    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec66c98-8eed-45cd-87ed-be4cce7a973f-kube-api-access-8fzx8" (OuterVolumeSpecName: "kube-api-access-8fzx8") pod "bec66c98-8eed-45cd-87ed-be4cce7a973f" (UID: "bec66c98-8eed-45cd-87ed-be4cce7a973f"). InnerVolumeSpecName "kube-api-access-8fzx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:30:05.121179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.121146    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:30:05.121179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.121173    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec66c98-8eed-45cd-87ed-be4cce7a973f-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:30:05.121179 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.121183    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fzx8\" (UniqueName: \"kubernetes.io/projected/bec66c98-8eed-45cd-87ed-be4cce7a973f-kube-api-access-8fzx8\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:30:05.283220 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.283189    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"]
Sep 18 18:30:05.287753 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:05.287722    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-12b560-5cc75f58d6-bgnfr"]
Sep 18 18:30:06.921618 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:06.921255    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bec66c98-8eed-45cd-87ed-be4cce7a973f" path="/var/lib/kubelet/pods/bec66c98-8eed-45cd-87ed-be4cce7a973f/volumes"
Sep 18 18:30:19.026176 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:19.026150    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:30:19.034315 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:19.034295    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:30:23.069292 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:23.069258    2582 generic.go:356] "Generic (PLEG): container finished" podID="944e07c9-b553-45bc-8438-2989850f4016" containerID="88a639a9ce6076b14128ea42882d114a31892a8c8bbceb3054942366dfdb1195" exitCode=1
Sep 18 18:30:23.069679 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:23.069328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn" event={"ID":"944e07c9-b553-45bc-8438-2989850f4016","Type":"ContainerDied","Data":"88a639a9ce6076b14128ea42882d114a31892a8c8bbceb3054942366dfdb1195"}
Sep 18 18:30:23.069720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:23.069700    2582 scope.go:117] "RemoveContainer" containerID="88a639a9ce6076b14128ea42882d114a31892a8c8bbceb3054942366dfdb1195"
Sep 18 18:30:23.070000 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:23.069986    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:30:24.077526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:24.077492    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn" event={"ID":"944e07c9-b553-45bc-8438-2989850f4016","Type":"ContainerStarted","Data":"cb25db7e0ba2ce14addef64cc01c65cfe145a4db388e40eae3f1295718993cb4"}
Sep 18 18:30:24.078017 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:24.077704    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:30:35.085260 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:30:35.085231    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rtwmn"
Sep 18 18:31:40.278880 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.278841    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"]
Sep 18 18:31:40.279518 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.279122    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" podUID="77a2c67f-f184-4828-944f-114a39586056" containerName="maas-api" containerID="cri-o://efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584" gracePeriod=30
Sep 18 18:31:40.314695 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.314668    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"]
Sep 18 18:31:40.317083 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.317059    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-46c63c-29829270-gxvf9"]
Sep 18 18:31:40.545802 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.545776    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:31:40.571787 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.571758    2582 generic.go:356] "Generic (PLEG): container finished" podID="77a2c67f-f184-4828-944f-114a39586056" containerID="efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584" exitCode=0
Sep 18 18:31:40.571928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.571800    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"
Sep 18 18:31:40.571928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.571802    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" event={"ID":"77a2c67f-f184-4828-944f-114a39586056","Type":"ContainerDied","Data":"efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584"}
Sep 18 18:31:40.571928 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.571920    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp" event={"ID":"77a2c67f-f184-4828-944f-114a39586056","Type":"ContainerDied","Data":"29e01b0099a81621afce45b072799b78bda164fbbbbbfd6e61a4750b61ae467a"}
Sep 18 18:31:40.572086 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.571946    2582 scope.go:117] "RemoveContainer" containerID="efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584"
Sep 18 18:31:40.581968 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.581954    2582 scope.go:117] "RemoveContainer" containerID="efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584"
Sep 18 18:31:40.582307 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:31:40.582284    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584\": container with ID starting with efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584 not found: ID does not exist" containerID="efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584"
Sep 18 18:31:40.582402 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.582318    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584"} err="failed to get container status \"efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584\": rpc error: code = NotFound desc = could not find container \"efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584\": container with ID starting with efc3c18f2373934c62d79829b771dacdd2b63daf33d4539dbafaafbcdc424584 not found: ID does not exist"
Sep 18 18:31:40.632035 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.632005    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls\") pod \"77a2c67f-f184-4828-944f-114a39586056\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") "
Sep 18 18:31:40.632245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.632132    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77a2c67f-f184-4828-944f-114a39586056-openshift-service-ca\") pod \"77a2c67f-f184-4828-944f-114a39586056\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") "
Sep 18 18:31:40.632245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.632198    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbs6r\" (UniqueName: \"kubernetes.io/projected/77a2c67f-f184-4828-944f-114a39586056-kube-api-access-zbs6r\") pod \"77a2c67f-f184-4828-944f-114a39586056\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") "
Sep 18 18:31:40.632245 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.632223    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-metrics-certs\") pod \"77a2c67f-f184-4828-944f-114a39586056\" (UID: \"77a2c67f-f184-4828-944f-114a39586056\") "
Sep 18 18:31:40.632541 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.632509    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77a2c67f-f184-4828-944f-114a39586056-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77a2c67f-f184-4828-944f-114a39586056" (UID: "77a2c67f-f184-4828-944f-114a39586056"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:31:40.634402 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.634378    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77a2c67f-f184-4828-944f-114a39586056" (UID: "77a2c67f-f184-4828-944f-114a39586056"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:31:40.634511 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.634444    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a2c67f-f184-4828-944f-114a39586056-kube-api-access-zbs6r" (OuterVolumeSpecName: "kube-api-access-zbs6r") pod "77a2c67f-f184-4828-944f-114a39586056" (UID: "77a2c67f-f184-4828-944f-114a39586056"). InnerVolumeSpecName "kube-api-access-zbs6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:31:40.634511 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.634460    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "77a2c67f-f184-4828-944f-114a39586056" (UID: "77a2c67f-f184-4828-944f-114a39586056"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:31:40.733797 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.733753    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:40.733797 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.733791    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77a2c67f-f184-4828-944f-114a39586056-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:40.733797 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.733804    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbs6r\" (UniqueName: \"kubernetes.io/projected/77a2c67f-f184-4828-944f-114a39586056-kube-api-access-zbs6r\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:40.734037 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.733816    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77a2c67f-f184-4828-944f-114a39586056-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:40.894476 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.894446    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"]
Sep 18 18:31:40.900609 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.900579    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-46c63c-6854c65bff-nwtsp"]
Sep 18 18:31:40.920909 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.920881    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a2c67f-f184-4828-944f-114a39586056" path="/var/lib/kubelet/pods/77a2c67f-f184-4828-944f-114a39586056/volumes"
Sep 18 18:31:40.921235 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:40.921221    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba" path="/var/lib/kubelet/pods/8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba/volumes"
Sep 18 18:31:57.944085 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:57.944037    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"]
Sep 18 18:31:57.944506 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:57.944257    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" podUID="65732db1-93bc-4dfb-9fdc-b0e3770b93ba" containerName="maas-api" containerID="cri-o://0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e" gracePeriod=30
Sep 18 18:31:58.189244 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.189219    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:31:58.310795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.310709    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls\") pod \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") "
Sep 18 18:31:58.310795 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.310765    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-metrics-certs\") pod \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") "
Sep 18 18:31:58.311019 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.310876    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-openshift-service-ca\") pod \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") "
Sep 18 18:31:58.311019 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.310945    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-946g9\" (UniqueName: \"kubernetes.io/projected/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-kube-api-access-946g9\") pod \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\" (UID: \"65732db1-93bc-4dfb-9fdc-b0e3770b93ba\") "
Sep 18 18:31:58.311279 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.311246    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65732db1-93bc-4dfb-9fdc-b0e3770b93ba" (UID: "65732db1-93bc-4dfb-9fdc-b0e3770b93ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:31:58.313123 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.313099    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65732db1-93bc-4dfb-9fdc-b0e3770b93ba" (UID: "65732db1-93bc-4dfb-9fdc-b0e3770b93ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:31:58.313237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.313199    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "65732db1-93bc-4dfb-9fdc-b0e3770b93ba" (UID: "65732db1-93bc-4dfb-9fdc-b0e3770b93ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:31:58.313237 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.313141    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-kube-api-access-946g9" (OuterVolumeSpecName: "kube-api-access-946g9") pod "65732db1-93bc-4dfb-9fdc-b0e3770b93ba" (UID: "65732db1-93bc-4dfb-9fdc-b0e3770b93ba"). InnerVolumeSpecName "kube-api-access-946g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:31:58.411809 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.411771    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:58.411809 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.411802    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:58.411809 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.411814    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:58.412076 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.411828    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-946g9\" (UniqueName: \"kubernetes.io/projected/65732db1-93bc-4dfb-9fdc-b0e3770b93ba-kube-api-access-946g9\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:31:58.690390 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.690294    2582 generic.go:356] "Generic (PLEG): container finished" podID="65732db1-93bc-4dfb-9fdc-b0e3770b93ba" containerID="0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e" exitCode=0
Sep 18 18:31:58.690390 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.690373    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"
Sep 18 18:31:58.690676 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.690465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" event={"ID":"65732db1-93bc-4dfb-9fdc-b0e3770b93ba","Type":"ContainerDied","Data":"0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e"}
Sep 18 18:31:58.690676 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.690520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97" event={"ID":"65732db1-93bc-4dfb-9fdc-b0e3770b93ba","Type":"ContainerDied","Data":"02a78323a75aa956c1cb1a0632d63635eb552ccfc6613a76ba2aa7875cd7bd29"}
Sep 18 18:31:58.690676 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.690550    2582 scope.go:117] "RemoveContainer" containerID="0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e"
Sep 18 18:31:58.705633 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.705614    2582 scope.go:117] "RemoveContainer" containerID="0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e"
Sep 18 18:31:58.705944 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:31:58.705916    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e\": container with ID starting with 0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e not found: ID does not exist" containerID="0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e"
Sep 18 18:31:58.705990 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.705956    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e"} err="failed to get container status \"0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e\": rpc error: code = NotFound desc = could not find container \"0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e\": container with ID starting with 0a8c59bec2fcbd4e1ac324eabef761eb0c3ccaa6fc3f51aa37af7a81dc21ec9e not found: ID does not exist"
Sep 18 18:31:58.717900 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.717863    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"]
Sep 18 18:31:58.721462 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.721437    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-011541-5cfc9669c-nhx97"]
Sep 18 18:31:58.920542 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:31:58.920508    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65732db1-93bc-4dfb-9fdc-b0e3770b93ba" path="/var/lib/kubelet/pods/65732db1-93bc-4dfb-9fdc-b0e3770b93ba/volumes"
Sep 18 18:32:15.512825 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.512783    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"]
Sep 18 18:32:15.513977 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.513646    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" podUID="929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" containerName="maas-api" containerID="cri-o://6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e" gracePeriod=30
Sep 18 18:32:15.765390 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.765330    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:32:15.773948 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.773925    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls\") pod \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") "
Sep 18 18:32:15.774123 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.773959    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-metrics-certs\") pod \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") "
Sep 18 18:32:15.774123 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.773982    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-openshift-service-ca\") pod \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") "
Sep 18 18:32:15.774123 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.774007    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9nmt\" (UniqueName: \"kubernetes.io/projected/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-kube-api-access-w9nmt\") pod \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\" (UID: \"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c\") "
Sep 18 18:32:15.774381 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.774354    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" (UID: "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:32:15.776349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.776320    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-kube-api-access-w9nmt" (OuterVolumeSpecName: "kube-api-access-w9nmt") pod "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" (UID: "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c"). InnerVolumeSpecName "kube-api-access-w9nmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:32:15.776458 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.776346    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" (UID: "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:32:15.776458 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.776397    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" (UID: "929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:32:15.797139 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.797105    2582 generic.go:356] "Generic (PLEG): container finished" podID="929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" containerID="6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e" exitCode=0
Sep 18 18:32:15.797281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.797140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" event={"ID":"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c","Type":"ContainerDied","Data":"6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e"}
Sep 18 18:32:15.797281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.797173    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"
Sep 18 18:32:15.797281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.797218    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x" event={"ID":"929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c","Type":"ContainerDied","Data":"c176d61b5061d9845fb998008812726f6e39fb26c1454b25eb9406be38a6ca66"}
Sep 18 18:32:15.797281 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.797250    2582 scope.go:117] "RemoveContainer" containerID="6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e"
Sep 18 18:32:15.807942 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.807922    2582 scope.go:117] "RemoveContainer" containerID="6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e"
Sep 18 18:32:15.808275 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:32:15.808236    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e\": container with ID starting with 6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e not found: ID does not exist" containerID="6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e"
Sep 18 18:32:15.808375 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.808283    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e"} err="failed to get container status \"6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e\": rpc error: code = NotFound desc = could not find container \"6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e\": container with ID starting with 6687d9465066ea99de15f73e85657dee9dc0da0039beb4010113890a3f97fa9e not found: ID does not exist"
Sep 18 18:32:15.820829 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.820798    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"]
Sep 18 18:32:15.824044 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.824022    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99602c-7d95d49f4f-v294x"]
Sep 18 18:32:15.875351 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.875323    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:15.875351 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.875350    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:15.875522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.875361    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:15.875522 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:15.875370    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9nmt\" (UniqueName: \"kubernetes.io/projected/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c-kube-api-access-w9nmt\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:16.922198 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:16.922163    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" path="/var/lib/kubelet/pods/929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c/volumes"
Sep 18 18:32:24.638970 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:24.638924    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc7cc4dff-q27pw"]
Sep 18 18:32:24.639526 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:24.639203    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw" podUID="809f1117-84f1-498d-87fd-a163500439cd" containerName="manager" containerID="cri-o://6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df" gracePeriod=30
Sep 18 18:32:24.987829 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:24.987789    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:32:25.063211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.063176    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-webhook-cert\") pod \"809f1117-84f1-498d-87fd-a163500439cd\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") "
Sep 18 18:32:25.063377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.063249    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-metrics-certs\") pod \"809f1117-84f1-498d-87fd-a163500439cd\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") "
Sep 18 18:32:25.063377 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.063358    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ltwt\" (UniqueName: \"kubernetes.io/projected/809f1117-84f1-498d-87fd-a163500439cd-kube-api-access-7ltwt\") pod \"809f1117-84f1-498d-87fd-a163500439cd\" (UID: \"809f1117-84f1-498d-87fd-a163500439cd\") "
Sep 18 18:32:25.065404 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.065374    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "809f1117-84f1-498d-87fd-a163500439cd" (UID: "809f1117-84f1-498d-87fd-a163500439cd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:32:25.065566 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.065550    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809f1117-84f1-498d-87fd-a163500439cd-kube-api-access-7ltwt" (OuterVolumeSpecName: "kube-api-access-7ltwt") pod "809f1117-84f1-498d-87fd-a163500439cd" (UID: "809f1117-84f1-498d-87fd-a163500439cd"). InnerVolumeSpecName "kube-api-access-7ltwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:32:25.065844 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.065829    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "809f1117-84f1-498d-87fd-a163500439cd" (UID: "809f1117-84f1-498d-87fd-a163500439cd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:32:25.164339 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.164243    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ltwt\" (UniqueName: \"kubernetes.io/projected/809f1117-84f1-498d-87fd-a163500439cd-kube-api-access-7ltwt\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:25.164339 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.164279    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-webhook-cert\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:25.164339 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.164293    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/809f1117-84f1-498d-87fd-a163500439cd-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:32:25.860309 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.860273    2582 generic.go:356] "Generic (PLEG): container finished" podID="809f1117-84f1-498d-87fd-a163500439cd" containerID="6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df" exitCode=0
Sep 18 18:32:25.860719 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.860324    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw"
Sep 18 18:32:25.860719 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.860353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw" event={"ID":"809f1117-84f1-498d-87fd-a163500439cd","Type":"ContainerDied","Data":"6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df"}
Sep 18 18:32:25.860719 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.860406    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc7cc4dff-q27pw" event={"ID":"809f1117-84f1-498d-87fd-a163500439cd","Type":"ContainerDied","Data":"6400fe65fc5850c88fe67db9baff7fdd6c2ad1c4f88bfc3d7574a7cbea512e71"}
Sep 18 18:32:25.860719 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.860428    2582 scope.go:117] "RemoveContainer" containerID="6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df"
Sep 18 18:32:25.872121 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.872035    2582 scope.go:117] "RemoveContainer" containerID="6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df"
Sep 18 18:32:25.872391 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:32:25.872370    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df\": container with ID starting with 6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df not found: ID does not exist" containerID="6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df"
Sep 18 18:32:25.872459 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.872403    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df"} err="failed to get container status \"6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df\": rpc error: code = NotFound desc = could not find container \"6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df\": container with ID starting with 6517e58623c4e8bfbab99ba8091b4d007e08cca6e8791b9c40dd1d0c03b476df not found: ID does not exist"
Sep 18 18:32:25.883403 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.883380    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc7cc4dff-q27pw"]
Sep 18 18:32:25.887291 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:25.887268    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dc7cc4dff-q27pw"]
Sep 18 18:32:26.920159 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:32:26.920127    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="809f1117-84f1-498d-87fd-a163500439cd" path="/var/lib/kubelet/pods/809f1117-84f1-498d-87fd-a163500439cd/volumes"
Sep 18 18:35:19.072074 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:35:19.072017    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:35:19.077996 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:35:19.077971    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:36:19.313003 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:36:19.312968    2582 scope.go:117] "RemoveContainer" containerID="a6697ad097a5348e07c46b0b82e4218a40bf4b4977e99939e34c22f736242a7d"
Sep 18 18:37:33.796677 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.796640    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"]
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797040    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a2c67f-f184-4828-944f-114a39586056" containerName="maas-api"
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797067    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a2c67f-f184-4828-944f-114a39586056" containerName="maas-api"
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797079    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba" containerName="cleanup"
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797084    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba" containerName="cleanup"
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797094    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec66c98-8eed-45cd-87ed-be4cce7a973f" containerName="maas-api"
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797101    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec66c98-8eed-45cd-87ed-be4cce7a973f" containerName="maas-api"
Sep 18 18:37:33.797112 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797112    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65732db1-93bc-4dfb-9fdc-b0e3770b93ba" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797117    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65732db1-93bc-4dfb-9fdc-b0e3770b93ba" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797129    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="809f1117-84f1-498d-87fd-a163500439cd" containerName="manager"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797134    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="809f1117-84f1-498d-87fd-a163500439cd" containerName="manager"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797145    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797150    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797224    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a2c67f-f184-4828-944f-114a39586056" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797233    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec66c98-8eed-45cd-87ed-be4cce7a973f" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797239    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="929563e7-7caf-4dd7-bb6b-4ca7f3a62a1c" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797245    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65732db1-93bc-4dfb-9fdc-b0e3770b93ba" containerName="maas-api"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797250    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="809f1117-84f1-498d-87fd-a163500439cd" containerName="manager"
Sep 18 18:37:33.797334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.797255    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff793fd-bac4-4d2f-9bd2-5e6c631a8aba" containerName="cleanup"
Sep 18 18:37:33.800554 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.800527    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.803810 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.803781    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 18:37:33.803925 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.803784    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jkbfn\""
Sep 18 18:37:33.803925 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.803814    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:37:33.809319 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.809301    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"]
Sep 18 18:37:33.814097 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.814071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-maas-api-tls\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.814248 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.814227    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-metrics-certs\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.814316 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.814271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-openshift-service-ca\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.814370 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.814336    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqc52\" (UniqueName: \"kubernetes.io/projected/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-kube-api-access-vqc52\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.914775 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.914748    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-metrics-certs\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.914938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.914783    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-openshift-service-ca\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.914938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.914808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqc52\" (UniqueName: \"kubernetes.io/projected/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-kube-api-access-vqc52\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.914938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.914833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-maas-api-tls\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.915431 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.915408    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-openshift-service-ca\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.917430 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.917402    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-metrics-certs\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.917586 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.917564    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-maas-api-tls\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:33.922581 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:33.922564    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqc52\" (UniqueName: \"kubernetes.io/projected/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-kube-api-access-vqc52\") pod \"maas-api-59f6566bbc-4xs28\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:34.113186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:34.113099    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:34.391383 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:34.389920    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"]
Sep 18 18:37:34.395862 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:34.395786    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:37:34.721098 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:34.721001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" event={"ID":"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac","Type":"ContainerStarted","Data":"ed7486eefc9660c638ec3b95227d7f922ae614b56962fbea29d3389e9bef2b17"}
Sep 18 18:37:35.728186 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:35.728151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" event={"ID":"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac","Type":"ContainerStarted","Data":"52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283"}
Sep 18 18:37:35.728582 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:35.728282    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:35.745705 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:35.745651    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" podStartSLOduration=1.959073442 podStartE2EDuration="2.745633046s" totalImagesPullingTime="786.559604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:37:35.745490792 +0000 UTC m=+2239.172703546" watchObservedRunningTime="2026-09-18 18:37:35.745633046 +0000 UTC m=+2239.172845797"
Sep 18 18:37:41.739161 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:41.739133    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:51.456097 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:51.456043    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"]
Sep 18 18:37:51.456641 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:51.456354    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" podUID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" containerName="maas-api" containerID="cri-o://52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283" gracePeriod=30
Sep 18 18:37:51.733901 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:51.733809    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" podUID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.88:8443/health\": dial tcp 10.132.0.88:8443: connect: connection refused"
Sep 18 18:37:52.709533 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.709510    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:52.803973 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.803941    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-metrics-certs\") pod \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") "
Sep 18 18:37:52.804167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.803986    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-openshift-service-ca\") pod \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") "
Sep 18 18:37:52.804167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.804009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqc52\" (UniqueName: \"kubernetes.io/projected/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-kube-api-access-vqc52\") pod \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") "
Sep 18 18:37:52.804167 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.804123    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-maas-api-tls\") pod \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\" (UID: \"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac\") "
Sep 18 18:37:52.804389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.804334    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" (UID: "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:37:52.804456 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.804419    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-openshift-service-ca\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:37:52.806252 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.806227    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" (UID: "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:37:52.806372 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.806312    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-kube-api-access-vqc52" (OuterVolumeSpecName: "kube-api-access-vqc52") pod "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" (UID: "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac"). InnerVolumeSpecName "kube-api-access-vqc52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:37:52.806427 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.806372    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" (UID: "7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:37:52.831737 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.831709    2582 generic.go:356] "Generic (PLEG): container finished" podID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" containerID="52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283" exitCode=0
Sep 18 18:37:52.831842 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.831756    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"
Sep 18 18:37:52.831842 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.831800    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" event={"ID":"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac","Type":"ContainerDied","Data":"52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283"}
Sep 18 18:37:52.831916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.831865    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28" event={"ID":"7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac","Type":"ContainerDied","Data":"ed7486eefc9660c638ec3b95227d7f922ae614b56962fbea29d3389e9bef2b17"}
Sep 18 18:37:52.831916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.831894    2582 scope.go:117] "RemoveContainer" containerID="52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283"
Sep 18 18:37:52.842969 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.842951    2582 scope.go:117] "RemoveContainer" containerID="52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283"
Sep 18 18:37:52.843271 ip-10-0-135-209 kubenswrapper[2582]: E0918 18:37:52.843253    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283\": container with ID starting with 52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283 not found: ID does not exist" containerID="52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283"
Sep 18 18:37:52.843334 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.843280    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283"} err="failed to get container status \"52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283\": rpc error: code = NotFound desc = could not find container \"52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283\": container with ID starting with 52e944361542483d8481a319aeb8d73c5b57c9d5921baee41a8e8c2ae5c55283 not found: ID does not exist"
Sep 18 18:37:52.855542 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.855431    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"]
Sep 18 18:37:52.858256 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.858237    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-4xs28"]
Sep 18 18:37:52.905906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.905882    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-maas-api-tls\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:37:52.905906 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.905903    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-metrics-certs\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:37:52.906033 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.905911    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqc52\" (UniqueName: \"kubernetes.io/projected/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac-kube-api-access-vqc52\") on node \"ip-10-0-135-209.ec2.internal\" DevicePath \"\""
Sep 18 18:37:52.920611 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:37:52.920587    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" path="/var/lib/kubelet/pods/7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac/volumes"
Sep 18 18:38:07.249876 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:07.249841    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8grrr_d6a1fbbd-d154-4857-9e94-f4a938082b6c/global-pull-secret-syncer/0.log"
Sep 18 18:38:07.431044 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:07.431011    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pdzvz_b0445beb-a1ca-4b60-be4a-8f040edeab39/konnectivity-agent/0.log"
Sep 18 18:38:07.503811 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:07.503738    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-209.ec2.internal_cccdef02f8e26bf3341296b9df4a415f/haproxy/0.log"
Sep 18 18:38:12.212231 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:12.212144    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-v4qkj_60d3af17-9608-4189-a31c-39e85ebf883f/manager/0.log"
Sep 18 18:38:12.235724 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:12.235699    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-6pnvq_f111d43f-fb09-4686-b4d8-1277a721ba1d/manager/0.log"
Sep 18 18:38:13.894720 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:13.894692    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-hxhc4_6a3558a4-49fc-4e96-986f-c1a7821da11a/cluster-monitoring-operator/0.log"
Sep 18 18:38:13.922758 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:13.922733    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-rcc59_7f4117fc-34f7-4742-96c6-88a1e481a63d/kube-state-metrics/0.log"
Sep 18 18:38:13.940389 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:13.940363    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-rcc59_7f4117fc-34f7-4742-96c6-88a1e481a63d/kube-rbac-proxy-main/0.log"
Sep 18 18:38:13.961478 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:13.961460    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-rcc59_7f4117fc-34f7-4742-96c6-88a1e481a63d/kube-rbac-proxy-self/0.log"
Sep 18 18:38:14.010075 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.010027    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-d8n5h_53153c7a-a902-474b-b7f7-2af4ef8461f6/monitoring-plugin/0.log"
Sep 18 18:38:14.107677 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.107649    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mkphf_8a6d5da1-0016-409e-a3a8-203938e69149/node-exporter/0.log"
Sep 18 18:38:14.129180 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.129157    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mkphf_8a6d5da1-0016-409e-a3a8-203938e69149/kube-rbac-proxy/0.log"
Sep 18 18:38:14.150256 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.150200    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mkphf_8a6d5da1-0016-409e-a3a8-203938e69149/init-textfile/0.log"
Sep 18 18:38:14.565412 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.565382    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-jx4j6_b99311ed-0547-4c7c-a417-ba5ec0a32d8f/prometheus-operator/0.log"
Sep 18 18:38:14.587164 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.587141    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-jx4j6_b99311ed-0547-4c7c-a417-ba5ec0a32d8f/kube-rbac-proxy/0.log"
Sep 18 18:38:14.673004 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.672970    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bf777bb65-fphk8_46e4eef9-a010-445e-a81b-5f7489785a31/telemeter-client/0.log"
Sep 18 18:38:14.697356 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.697331    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bf777bb65-fphk8_46e4eef9-a010-445e-a81b-5f7489785a31/reload/0.log"
Sep 18 18:38:14.718895 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:14.718871    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bf777bb65-fphk8_46e4eef9-a010-445e-a81b-5f7489785a31/kube-rbac-proxy/0.log"
Sep 18 18:38:16.308798 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:16.308767    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-g47n6_b2b950ab-bdf2-4d50-8467-d4d2e762af1f/networking-console-plugin/0.log"
Sep 18 18:38:17.079349 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.079320    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/1.log"
Sep 18 18:38:17.085490 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.085465    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-w8lzm_c19150e6-187a-4e99-907e-b584d8b3aa55/console-operator/2.log"
Sep 18 18:38:17.353922 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.353839    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"]
Sep 18 18:38:17.354335 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.354280    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" containerName="maas-api"
Sep 18 18:38:17.354335 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.354292    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" containerName="maas-api"
Sep 18 18:38:17.354417 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.354390    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4d3ef4-00ae-41a6-a40c-e5b67ba84fac" containerName="maas-api"
Sep 18 18:38:17.357668 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.357517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.360183 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.360160    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8m5v\"/\"openshift-service-ca.crt\""
Sep 18 18:38:17.361247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.361223    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8m5v\"/\"kube-root-ca.crt\""
Sep 18 18:38:17.361380 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.361356    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x8m5v\"/\"default-dockercfg-qmjsc\""
Sep 18 18:38:17.364457 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.364435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"]
Sep 18 18:38:17.429703 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.429666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-lib-modules\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.429894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.429722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2hf2\" (UniqueName: \"kubernetes.io/projected/1a019cf3-24dc-4859-bd64-78498620bf0b-kube-api-access-b2hf2\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.429894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.429772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-proc\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.429894 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.429866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-sys\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.430078 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.429906    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-podres\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.530855 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.530820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-proc\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.530893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-sys\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.530935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-podres\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.530947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-proc\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.530997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-lib-modules\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531026 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.531022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2hf2\" (UniqueName: \"kubernetes.io/projected/1a019cf3-24dc-4859-bd64-78498620bf0b-kube-api-access-b2hf2\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.531088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-podres\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.531000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-sys\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.531310 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.531105    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a019cf3-24dc-4859-bd64-78498620bf0b-lib-modules\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.539043 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.539009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2hf2\" (UniqueName: \"kubernetes.io/projected/1a019cf3-24dc-4859-bd64-78498620bf0b-kube-api-access-b2hf2\") pod \"perf-node-gather-daemonset-gl47b\" (UID: \"1a019cf3-24dc-4859-bd64-78498620bf0b\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.629647 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.629559    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c88668489-fngp8_df945b42-5456-4a33-85c8-420ab4bf72ad/console/0.log"
Sep 18 18:38:17.669899 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.669871    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:17.942718 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.942687    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"]
Sep 18 18:38:17.980613 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:17.980584    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b" event={"ID":"1a019cf3-24dc-4859-bd64-78498620bf0b","Type":"ContainerStarted","Data":"5c054d1a2b993c63de27dc3a68f4d9ae02dcfc5d425ca9f809956b830c7ec5fc"}
Sep 18 18:38:18.486470 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:18.486440    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-ddlj2_65231b2f-8527-44cc-9bc1-6629067f9311/volume-data-source-validator/0.log"
Sep 18 18:38:18.988213 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:18.988171    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b" event={"ID":"1a019cf3-24dc-4859-bd64-78498620bf0b","Type":"ContainerStarted","Data":"0af40dd2fe79de9ba64233756a1cbed65c2cfc49a9e6f794d6a57dae19b8242d"}
Sep 18 18:38:18.988402 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:18.988242    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:19.010727 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:19.010680    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b" podStartSLOduration=2.01066585 podStartE2EDuration="2.01066585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:38:19.008130716 +0000 UTC m=+2282.435343471" watchObservedRunningTime="2026-09-18 18:38:19.01066585 +0000 UTC m=+2282.437878651"
Sep 18 18:38:19.698771 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:19.698680    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vwp4c_995ff60d-7dd5-491a-aca8-34ce1216a8bc/dns/0.log"
Sep 18 18:38:19.721822 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:19.721795    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vwp4c_995ff60d-7dd5-491a-aca8-34ce1216a8bc/kube-rbac-proxy/0.log"
Sep 18 18:38:19.787393 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:19.787366    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-trjfw_3dc8b275-9b48-462a-a66f-659147ccc56d/dns-node-resolver/0.log"
Sep 18 18:38:20.326253 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:20.326219    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6fz2m_dba2c0a8-d33d-4dc7-9b6b-0ec277193214/node-ca/0.log"
Sep 18 18:38:21.268467 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.268441    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6_a2cef883-e888-4392-ba52-2fe8fcbabe34/istio-proxy/0.log"
Sep 18 18:38:21.619321 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.619286    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cnvjt_93782abd-481e-457f-b67c-5469a42d428e/discovery/0.log"
Sep 18 18:38:21.640702 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.640677    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fbf455f6-rs4vh_e0fb050f-e042-4027-bfa4-474b12a84c68/kube-auth-proxy/0.log"
Sep 18 18:38:21.691169 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.691138    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tvsjk_7cc00339-98d4-4edd-8e78-8ebbae73a641/istio-proxy/0.log"
Sep 18 18:38:21.693634 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.693614    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tvsjk_7cc00339-98d4-4edd-8e78-8ebbae73a641/istio-proxy/1.log"
Sep 18 18:38:21.712190 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.712168    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-76f657cdc4-59w9r_1150832c-dd30-44ad-9a79-1bf56352abf0/payload-pre-processing/0.log"
Sep 18 18:38:21.739690 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.739669    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:38:21.771813 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:21.771792    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-598c64547d-8cwhs_ca87bb38-f503-4312-8ea0-ea8941f1ea67/router/0.log"
Sep 18 18:38:22.622262 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:22.622231    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bhppq_82d574cd-7c7e-48d3-9c56-af1bb63402b4/serve-healthcheck-canary/0.log"
Sep 18 18:38:23.099420 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:23.099388    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-tm2tl_4b17ce39-8f2d-4496-94d3-b62b3c984857/insights-operator/0.log"
Sep 18 18:38:23.100579 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:23.100559    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-tm2tl_4b17ce39-8f2d-4496-94d3-b62b3c984857/insights-operator/1.log"
Sep 18 18:38:23.190359 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:23.190330    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2k9s_4965f0d2-8952-4a08-b640-541fa3e20363/kube-rbac-proxy/0.log"
Sep 18 18:38:23.210293 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:23.210270    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2k9s_4965f0d2-8952-4a08-b640-541fa3e20363/exporter/0.log"
Sep 18 18:38:23.230731 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:23.230709    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2k9s_4965f0d2-8952-4a08-b640-541fa3e20363/extractor/0.log"
Sep 18 18:38:25.005348 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:25.005322    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-gl47b"
Sep 18 18:38:25.860463 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:25.860439    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rchwf_3c8f7f89-ccc4-4fc5-8a98-daa6ea58c806/server/0.log"
Sep 18 18:38:25.897577 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:25.897540    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rtwmn_944e07c9-b553-45bc-8438-2989850f4016/manager/1.log"
Sep 18 18:38:25.947648 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:25.947625    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rtwmn_944e07c9-b553-45bc-8438-2989850f4016/manager/0.log"
Sep 18 18:38:26.106036 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:26.106008    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57cbb766c6-dlxnk_5a41a42a-ec09-4a7b-8566-4cc36e482b9a/manager/0.log"
Sep 18 18:38:26.121664 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:26.121606    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57cbb766c6-dlxnk_5a41a42a-ec09-4a7b-8566-4cc36e482b9a/copy-manifests/0.log"
Sep 18 18:38:27.631961 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:27.631873    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jqfzz_7da96638-20d0-4ad0-8e28-2d73a223a718/openshift-lws-operator/0.log"
Sep 18 18:38:27.719835 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:27.719806    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6_a2cef883-e888-4392-ba52-2fe8fcbabe34/istio-proxy/0.log"
Sep 18 18:38:28.062247 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.062208    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cnvjt_93782abd-481e-457f-b67c-5469a42d428e/discovery/0.log"
Sep 18 18:38:28.070936 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.070907    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fbf455f6-rs4vh_e0fb050f-e042-4027-bfa4-474b12a84c68/kube-auth-proxy/0.log"
Sep 18 18:38:28.101355 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.101329    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tvsjk_7cc00339-98d4-4edd-8e78-8ebbae73a641/istio-proxy/0.log"
Sep 18 18:38:28.103836 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.103816    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tvsjk_7cc00339-98d4-4edd-8e78-8ebbae73a641/istio-proxy/1.log"
Sep 18 18:38:28.113887 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.113868    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-76f657cdc4-59w9r_1150832c-dd30-44ad-9a79-1bf56352abf0/payload-pre-processing/0.log"
Sep 18 18:38:28.129879 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.129851    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:38:28.150931 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:28.150894    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-598c64547d-8cwhs_ca87bb38-f503-4312-8ea0-ea8941f1ea67/router/0.log"
Sep 18 18:38:29.627712 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:29.627678    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8z7w6_a2cef883-e888-4392-ba52-2fe8fcbabe34/istio-proxy/0.log"
Sep 18 18:38:29.988887 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:29.988781    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cnvjt_93782abd-481e-457f-b67c-5469a42d428e/discovery/0.log"
Sep 18 18:38:30.005148 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:30.005122    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fbf455f6-rs4vh_e0fb050f-e042-4027-bfa4-474b12a84c68/kube-auth-proxy/0.log"
Sep 18 18:38:30.045703 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:30.045652    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tvsjk_7cc00339-98d4-4edd-8e78-8ebbae73a641/istio-proxy/0.log"
Sep 18 18:38:30.048592 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:30.048572    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tvsjk_7cc00339-98d4-4edd-8e78-8ebbae73a641/istio-proxy/1.log"
Sep 18 18:38:30.057608 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:30.057590    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-76f657cdc4-59w9r_1150832c-dd30-44ad-9a79-1bf56352abf0/payload-pre-processing/0.log"
Sep 18 18:38:30.073873 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:30.073816    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-85fd65c94c-ttfd5_4277a873-c982-4563-9f2c-306b90371db0/payload-processing/0.log"
Sep 18 18:38:30.091313 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:30.091296    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-598c64547d-8cwhs_ca87bb38-f503-4312-8ea0-ea8941f1ea67/router/0.log"
Sep 18 18:38:32.927861 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:32.927829    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-nht6h_38dfae07-7770-480a-941b-473232de5af7/kube-storage-version-migrator-operator/1.log"
Sep 18 18:38:32.928992 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:32.928972    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-nht6h_38dfae07-7770-480a-941b-473232de5af7/kube-storage-version-migrator-operator/0.log"
Sep 18 18:38:33.894936 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:33.894900    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6d2zm_fd8c8072-1646-495e-87c0-56da94590049/kube-multus/0.log"
Sep 18 18:38:34.189229 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.189152    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/kube-multus-additional-cni-plugins/0.log"
Sep 18 18:38:34.211916 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.211888    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/egress-router-binary-copy/0.log"
Sep 18 18:38:34.231882 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.231860    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/cni-plugins/0.log"
Sep 18 18:38:34.252372 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.252350    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/bond-cni-plugin/0.log"
Sep 18 18:38:34.271635 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.271608    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/routeoverride-cni/0.log"
Sep 18 18:38:34.289173 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.289143    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/whereabouts-cni-bincopy/0.log"
Sep 18 18:38:34.308211 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.308174    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7sqh6_0497d646-848f-4d71-b54c-db65e05579ee/whereabouts-cni/0.log"
Sep 18 18:38:34.527938 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.527850    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d9bp5_d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c/network-metrics-daemon/0.log"
Sep 18 18:38:34.545740 ip-10-0-135-209 kubenswrapper[2582]: I0918 18:38:34.545713    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d9bp5_d7c0c8d7-9c65-4a58-94e8-05506ecf9e7c/kube-rbac-proxy/0.log"