Sep 18 22:14:47.050989 ip-10-0-132-49 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 22:14:47.050999 ip-10-0-132-49 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 22:14:47.051008 ip-10-0-132-49 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 22:14:47.051323 ip-10-0-132-49 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 22:14:57.093756 ip-10-0-132-49 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 22:14:57.093778 ip-10-0-132-49 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 16226e9ae0ac4879a9e73975eeff73a3 --
Sep 18 22:16:54.139688 ip-10-0-132-49 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 22:16:57.457602 ip-10-0-132-49 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 22:16:57.457602 ip-10-0-132-49 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 22:16:57.457602 ip-10-0-132-49 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 22:16:57.457602 ip-10-0-132-49 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 22:16:57.457602 ip-10-0-132-49 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 22:16:57.458434 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.457721    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 22:16:57.462554 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462538    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 22:16:57.462554 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462552    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462556    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462559    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462562    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462564    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462567    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462570    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462573    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462575    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462582    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462585    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462587    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462590    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462592    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462595    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462598    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462602    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462604    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462607    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 22:16:57.462617 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462609    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462612    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462615    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462618    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462621    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462623    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462626    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462629    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462633    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462638    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462640    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462645    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462648    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462652    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462655    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462657    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462660    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462663    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462665    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 22:16:57.463067 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462668    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462671    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462673    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462677    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462680    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462682    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462685    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462687    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462690    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462692    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462695    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462698    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462700    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462703    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462705    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462710    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462713    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462716    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462719    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462722    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 22:16:57.463550 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462725    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462727    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462730    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462733    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462735    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462738    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462740    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462743    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462747    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462749    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462752    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462755    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462758    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462760    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462763    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462766    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462769    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462772    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462774    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462777    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 22:16:57.464128 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462780    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462783    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462785    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462788    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462790    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462793    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462795    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462799    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462802    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462804    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462807    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462811    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462815    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462818    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462821    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462824    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462826    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462829    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462832    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 22:16:57.464626 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.462834    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463526    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463533    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463536    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463539    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463542    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463545    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463548    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463551    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463554    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463557    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463560    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463563    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463566    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463569    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463572    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463575    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463578    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463581    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463584    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 22:16:57.465090 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463589    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463594    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463598    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463601    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463604    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463607    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463610    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463614    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463617    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463620    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463623    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463626    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463628    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463631    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463634    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463636    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463639    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463642    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463644    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463647    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 22:16:57.465589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463650    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463654    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463657    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463661    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463663    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463666    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463668    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463671    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463674    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463676    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463679    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463682    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463685    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463688    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463690    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463693    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463696    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463699    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463702    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463704    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 22:16:57.466085 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463707    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463710    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463712    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463715    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463718    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463721    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463725    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463727    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463730    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463733    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463735    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463738    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463741    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463743    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463746    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463749    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463752    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463755    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463758    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463761    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 22:16:57.466589 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463764    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463767    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463769    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463772    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463775    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463778    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463780    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463783    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463786    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463789    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463791    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463795    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463799    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463802    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463805    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463808    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463811    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463815    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463817    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 22:16:57.467071 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.463820    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464243    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464249    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464252    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464255    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464258    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464260    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464263    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464266    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464269    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464272    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464275    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464277    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464280    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464283    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464286    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464288    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464291    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464294    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464297    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 22:16:57.467559 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464299    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464302    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464305    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464307    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464310    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464312    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464315    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464318    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464321    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464323    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464326    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464328    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464331    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464333    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464336    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464339    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464341    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464344    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464346    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464349    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 22:16:57.468049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464351    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464354    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464357    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464359    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464362    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464364    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464367    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464371    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464374    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464380    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464398    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464401    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464403    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464408    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464410    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464413    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464416    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464419    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464421    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 22:16:57.468585 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464424    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464427    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464430    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464432    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464435    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464437    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464440    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464442    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464445    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464447    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464450    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464453    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464455    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464458    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464460    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464464    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464467    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464470    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464473    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464475    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 22:16:57.469049 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464478    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464481    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464485    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464489    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464492    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464494    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464497    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464500    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464502    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464506    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464510    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464513    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464515    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464518    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464520    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464523    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464526    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464528    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464531    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 22:16:57.469536 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464533    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.464537    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464608    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464654    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464661    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464665    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464670    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464673    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464677    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464681    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464686    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464689    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464692    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464696    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464699    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464704    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464707    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464710    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464714    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464717    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464721    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464723    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464727    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 22:16:57.470005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464730    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464734    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464738    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464741    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464744    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464747    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464751    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464754    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464757    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464761    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464764    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464768    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464771    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464774    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464777    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464780    2583 flags.go:64] FLAG: --enable-server="true"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464783    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464787    2583 flags.go:64] FLAG: --event-burst="100"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464790    2583 flags.go:64] FLAG: --event-qps="50"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464793    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464797    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464800    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464804    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464807    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464812    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 22:16:57.470579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464816    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464819    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464822    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464825    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464828    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464831    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464834    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464837    2583 flags.go:64] FLAG: --feature-gates=""
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464841    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464844    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464847    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464850    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464853    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464856    2583 flags.go:64] FLAG: --help="false"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464859    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464863    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464866    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464869    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464872    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464876    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464878    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464881    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464884    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 22:16:57.471172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464887    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464890    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464893    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464896    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464899    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464902    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464905    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464908    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464912    2583 flags.go:64] FLAG: --lock-file=""
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464915    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464918    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464922    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464927    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464930    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464933    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464936    2583 flags.go:64] FLAG: --logging-format="text"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464939    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464942    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464945    2583 flags.go:64] FLAG: --manifest-url=""
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464948    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464953    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464956    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464960    2583 flags.go:64] FLAG: --max-pods="110"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464962    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464965    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 22:16:57.471808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464968    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464972    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464975    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464978    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464981    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464988    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464991    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464994    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.464997    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465000    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465009    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465012    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465016    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465019    2583 flags.go:64] FLAG: --port="10250"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465022    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465025    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f3cc2ac2a310e3e4"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465030    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465033    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465036    2583 flags.go:64] FLAG: --register-node="true"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465039    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465047    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465050    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465053    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465056    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465060    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 22:16:57.472782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465063    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465067    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465070    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465072    2583 flags.go:64] FLAG: --runonce="false"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465075    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465079    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465082    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465085    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465088    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465091    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465095    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465098    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465101    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465104    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465107    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465110    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465113    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465116    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465119    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465125    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465127    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465131    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465135    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465140    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465143    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 22:16:57.473561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465146    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465149    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465152    2583 flags.go:64] FLAG: --v="2"
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465156    2583 flags.go:64] FLAG: --version="false"
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465159    2583 flags.go:64] FLAG: --vmodule=""
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465163    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.465166    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465292    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465297    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465300    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465303    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465306    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465309    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465312    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465315    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465332    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465336    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465339    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465342    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465345    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465348    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 22:16:57.474153 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465350    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465353    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465356    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465359    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465361    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465364    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465366    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465369    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465373    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465377    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465380    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465395    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465398    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465401    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465404    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465407    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465410    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465413    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465416    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465418    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 22:16:57.474681 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465421    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465424    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465427    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465429    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465432    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465435    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465437    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465440    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465443    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465446    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465448    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465451    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465454    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465456    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465459    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465462    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465464    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465467    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465469    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465472    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 22:16:57.475178 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465475    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465479    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465482    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465485    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465488    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465491    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465493    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465496    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465499    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465502    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465504    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465507    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465510    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465512    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465515    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465518    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465520    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465523    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465525    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 22:16:57.475678 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465528    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465532    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465536    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465539    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465542    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465546    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465550    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465553    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465556    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465559    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465561    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465566    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465569    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465572    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465580    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465583    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465586    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465589    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465592    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 22:16:57.476206 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465595    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465598    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465601    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465603    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465606    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465609    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.465611    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.466210    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.474524    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.474657    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.474683    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.474693    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474753    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474758    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474762    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 22:16:57.476684 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474766    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474769    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474772    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474774    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474777    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474780    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474783    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474786    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474789    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474792    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474794    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474797    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474799    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474809    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474812    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474814    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474817    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474819    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474822    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 22:16:57.477060 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474825    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474828    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474830    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474833    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474836    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474838    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474841    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474845    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474848    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474850    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474854    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474856    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474859    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474862    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474864    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474867    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474870    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474872    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474875    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474879    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 22:16:57.477538 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474883    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474886    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474889    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474892    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474894    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474897    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474900    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474904    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474909    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474912    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474916    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474919    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474922    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474925    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474928    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474930    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474933    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474936    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474938    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 22:16:57.478019 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474941    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474944    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474947    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474950    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474953    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474955    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474958    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474961    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474965    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474968    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474971    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474974    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474977    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474980    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474983    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474986    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474989    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474991    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474995    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 22:16:57.478488 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.474997    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475001    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475003    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475006    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475008    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475011    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475014    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475017    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475019    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475022    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475024    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475027    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475030    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475033    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475036    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475039    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475041    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475044    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 22:16:57.478945 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475047    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.475052    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475186    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475193    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475196    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475199    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475202    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475204    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475207    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475210    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475213    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475216    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475218    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475221    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475224    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 22:16:57.479451 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475227    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475229    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475232    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475234    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475237    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475240    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475242    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475245    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475248    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475251    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475253    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475256    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475259    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475263    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475267    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475270    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475273    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475276    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475279    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 22:16:57.479822 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475282    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475284    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475287    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475290    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475292    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475295    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475297    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475300    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475302    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475305    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475308    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475310    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475313    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475316    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475319    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475321    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475324    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475327    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475329    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 22:16:57.480287 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475333    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475337    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475339    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475342    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475344    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475348    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475350    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475353    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475356    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475358    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475361    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475364    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475366    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475369    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475372    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475374    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475377    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475380    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475399    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475404    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 22:16:57.480764 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475407    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475409    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475412    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475415    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475418    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475420    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475423    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475427    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475430    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475433    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475435    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475438    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475440    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475443    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475446    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475448    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475451    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475454    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475456    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475459    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 22:16:57.481237 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475462    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475465    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475467    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475470    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475474    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475478    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475480    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:57.475483    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.475488    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.475601    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.477781    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.478526    2583 server.go:1019] "Starting client certificate rotation"
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.478622    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 22:16:57.481773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.478660    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 22:16:57.482096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.480689    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 22:16:57.482096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.481864    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 22:16:57.498328 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.498310    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 18 22:16:57.499978 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.499961    2583 log.go:25] "Validated CRI v1 image API"
Sep 18 22:16:57.500854 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.500836    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 22:16:57.503496 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.503466    2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 dd1c07eb-0655-4c53-9d23-08892b87b2d8:/dev/nvme0n1p3 eb00c781-235f-4ccf-9fb7-042a7e7b1794:/dev/nvme0n1p4]
Sep 18 22:16:57.503566 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.503495    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 22:16:57.508571 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.508463    2583 manager.go:216] Machine: {Timestamp:2026-09-18 22:16:57.507961993 +0000 UTC m=+1.868478840 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099305 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ebc79b0ca3b0f3c557f8a063e1d4 SystemUUID:ec21ebc7-9b0c-a3b0-f3c5-57f8a063e1d4 BootID:16226e9a-e0ac-4879-a9e7-3975eeff73a3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ea:19:7d:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ea:19:7d:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:2f:1a:b5:32:d8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 22:16:57.508571 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.508561    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 22:16:57.508728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.508706    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 22:16:57.510128 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.510105    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 22:16:57.510261 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.510131    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-49.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 22:16:57.511284 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.511272    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 22:16:57.511337 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.511287    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 22:16:57.511337 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.511295    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 22:16:57.511337 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.511307    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 22:16:57.512599 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.512583    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 22:16:57.512667 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.512661    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 22:16:57.517030 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517018    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 22:16:57.517142 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517133    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 22:16:57.517180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517162    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 22:16:57.517180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517172    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 22:16:57.517238 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517186    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 22:16:57.517238 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517195    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 18 22:16:57.517238 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517203    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 22:16:57.517588 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517569    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 22:16:57.517693 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.517595    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 22:16:57.519409 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.519377    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 22:16:57.521739 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.521722    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 22:16:57.521814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.521753    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 22:16:57.522821 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522808    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 22:16:57.522863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522825    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 22:16:57.522863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522832    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 22:16:57.522863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522838    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 22:16:57.522863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522847    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 22:16:57.522863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522853    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522866    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522872    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522879    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522885    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522894    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522900    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522924    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 22:16:57.523025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.522930    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 22:16:57.525939 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.525906    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 22:16:57.525939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.525930    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-49.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 22:16:57.526081 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.526013    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 22:16:57.526941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.526925    2583 server.go:1268] "Started kubelet"
Sep 18 22:16:57.527060 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.526998    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 22:16:57.527158 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.527064    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 22:16:57.527158 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.527120    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 22:16:57.528415 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.528366    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 22:16:57.528505 ip-10-0-132-49 systemd[1]: Started Kubernetes Kubelet.
Sep 18 22:16:57.529086 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.529045    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 22:16:57.533227 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.533208    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 22:16:57.535252 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.535226    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 22:16:57.535438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.535407    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hwbqd"
Sep 18 22:16:57.535678 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.535665    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 22:16:57.535960 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.535937    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:57.536273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536258    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 22:16:57.536350 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536295    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 22:16:57.536350 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536332    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 22:16:57.536457 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536438    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 22:16:57.536524 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536510    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 22:16:57.536570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536545    2583 factory.go:55] Registering systemd factory
Sep 18 22:16:57.536570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536565    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 18 22:16:57.537058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536802    2583 factory.go:153] Registering CRI-O factory
Sep 18 22:16:57.537058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536817    2583 factory.go:223] Registration of the crio container factory successfully
Sep 18 22:16:57.537058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536882    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 22:16:57.537058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536902    2583 factory.go:103] Registering Raw factory
Sep 18 22:16:57.537058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.536918    2583 manager.go:1196] Started watching for new ooms in manager
Sep 18 22:16:57.537322 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.537309    2583 manager.go:349] Starting recovery of all containers
Sep 18 22:16:57.538210 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.538183    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 22:16:57.575555 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.575523    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 22:16:57.576070 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.575800    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hwbqd"
Sep 18 22:16:57.585283 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.585259    2583 manager.go:354] Recovery completed
Sep 18 22:16:57.589322 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.589309    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.591076 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591059    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.591135 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591094    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.591135 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591108    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.591669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591657    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 22:16:57.591669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591668    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.591782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591669    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 22:16:57.591782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.591702    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 22:16:57.592353 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.592328    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.592353 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.592352    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.592538 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.592366    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.593758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.593746    2583 policy_none.go:49] "None policy: Start"
Sep 18 22:16:57.593797 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.593761    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 22:16:57.593797 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.593771    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 22:16:57.595788 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.595777    2583 policy_none.go:47] "Start"
Sep 18 22:16:57.632031 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632007    2583 manager.go:337] "Starting Device Plugin manager"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.632057    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632067    2583 server.go:87] "Starting device plugin registration server"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632323    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632334    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632437    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632511    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.632520    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.633102    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 22:16:57.648914 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.633163    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:57.674911 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.674887    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 22:16:57.676061 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.676043    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 22:16:57.676143 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.676089    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 22:16:57.676143 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.676110    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 22:16:57.676207 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.676145    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 22:16:57.678090 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.678068    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:16:57.732918 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.732809    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.734003 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.733978    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.734104 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.734010    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.734104 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.734025    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.734104 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.734050    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.743202 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.743188    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.743251 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.743208    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-49.ec2.internal\": node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:57.759267 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.759242    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:57.776853 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.776829    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"]
Sep 18 22:16:57.776953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.776905    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.778541 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.778526    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.778632 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.778558    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.778632 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.778572    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.780986 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.780971    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.781112 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781098    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.781158 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781124    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.781708 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781694    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.781791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781714    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.781791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781741    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.781791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781751    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.781906 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781721    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.781906 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.781808    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.784073 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.784059    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.784144 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.784088    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 22:16:57.785775 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.785755    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 22:16:57.785908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.785789    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 22:16:57.785908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.785804    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Sep 18 22:16:57.808978 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.808959    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-49.ec2.internal\" not found" node="ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.813403 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.813375    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-49.ec2.internal\" not found" node="ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.838497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.838469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/479909f6ffdb4c7950f561521538164e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"479909f6ffdb4c7950f561521538164e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.838610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.838505    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/479909f6ffdb4c7950f561521538164e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"479909f6ffdb4c7950f561521538164e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.838610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.838528    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f394ebf0066db702356187a5a575a284-config\") pod \"kube-apiserver-proxy-ip-10-0-132-49.ec2.internal\" (UID: \"f394ebf0066db702356187a5a575a284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.859669 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.859643    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:57.939372 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.939342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f394ebf0066db702356187a5a575a284-config\") pod \"kube-apiserver-proxy-ip-10-0-132-49.ec2.internal\" (UID: \"f394ebf0066db702356187a5a575a284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.939372 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.939373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/479909f6ffdb4c7950f561521538164e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"479909f6ffdb4c7950f561521538164e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.939564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.939405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/479909f6ffdb4c7950f561521538164e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"479909f6ffdb4c7950f561521538164e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.939564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.939423    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f394ebf0066db702356187a5a575a284-config\") pod \"kube-apiserver-proxy-ip-10-0-132-49.ec2.internal\" (UID: \"f394ebf0066db702356187a5a575a284\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.939564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.939432    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/479909f6ffdb4c7950f561521538164e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"479909f6ffdb4c7950f561521538164e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.939564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:57.939433    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/479909f6ffdb4c7950f561521538164e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"479909f6ffdb4c7950f561521538164e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:57.960471 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:57.960447    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.061295 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.061220    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.112392 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.112371    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:58.116798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.116779    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:58.162042 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.162007    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.262596 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.262565    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.363113 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.363028    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.463653 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.463625    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.478795 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.478770    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 22:16:58.478934 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.478911    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 22:16:58.539052 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.538584    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:16:58.546826 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.546795    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:16:58.564681 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.564650    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.581816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.581788    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 22:11:57 +0000 UTC" deadline="2028-03-08 00:34:53.755520063 +0000 UTC"
Sep 18 22:16:58.581922 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.581817    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12866h17m55.173707477s"
Sep 18 22:16:58.665452 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.665418    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.760692 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:58.760654    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod479909f6ffdb4c7950f561521538164e.slice/crio-0ffd30029c346b4bf1601eb71b6bd752b800ab78b3331bdeb4c4fb42728ff9f1 WatchSource:0}: Error finding container 0ffd30029c346b4bf1601eb71b6bd752b800ab78b3331bdeb4c4fb42728ff9f1: Status 404 returned error can't find the container with id 0ffd30029c346b4bf1601eb71b6bd752b800ab78b3331bdeb4c4fb42728ff9f1
Sep 18 22:16:58.762432 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:16:58.762400    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf394ebf0066db702356187a5a575a284.slice/crio-f6902ce8e7b2163ab8f0ddb2575faf5d47fbd8103df23ffa9be0b81440d733e0 WatchSource:0}: Error finding container f6902ce8e7b2163ab8f0ddb2575faf5d47fbd8103df23ffa9be0b81440d733e0: Status 404 returned error can't find the container with id f6902ce8e7b2163ab8f0ddb2575faf5d47fbd8103df23ffa9be0b81440d733e0
Sep 18 22:16:58.765502 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:58.765482    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Sep 18 22:16:58.767299 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.767284    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:16:58.840132 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.840101    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:16:58.936608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.936529    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:58.949589 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.949569    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 22:16:58.950412 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.950400    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Sep 18 22:16:58.959810 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:58.959793    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 22:16:59.519121 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.519092    2583 apiserver.go:52] "Watching apiserver"
Sep 18 22:16:59.526918 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.526880    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 22:16:59.527282 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.527251    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-km2gr","kube-system/global-pull-secret-syncer-4t6gh","kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6","openshift-cluster-node-tuning-operator/tuned-5p2xm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal","openshift-multus/multus-jwgtn","kube-system/konnectivity-agent-jst6w","openshift-dns/node-resolver-r28kg","openshift-image-registry/node-ca-gdk85","openshift-multus/multus-additional-cni-plugins-qjr2x","openshift-multus/network-metrics-daemon-xzzt2","openshift-network-diagnostics/network-check-target-lwvjn","openshift-network-operator/iptables-alerter-4nvgh"]
Sep 18 22:16:59.529897 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.529872    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.532414 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.532065    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:16:59.532414 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.532143    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:16:59.532785 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.532762    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jg4r5\""
Sep 18 22:16:59.533023 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.533001    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 22:16:59.533446 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.533191    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 22:16:59.533878 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.533701    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.533878 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.533827    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.535253 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.535234    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.535639 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.535582    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 22:16:59.537480 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.537462    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.539901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.539445    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 22:16:59.539901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.539499    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.539901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.539505    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.539901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.539753    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-867hh\""
Sep 18 22:16:59.539901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.539824    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.539901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.539842    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 22:16:59.540239 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.540022    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.540239 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.540071    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-95qgb\""
Sep 18 22:16:59.540602 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.540582    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.542099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.542079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.543544 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.543196    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.543544 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.543421    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zwjfq\""
Sep 18 22:16:59.543854 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.543834    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.544416 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.544396    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.545499 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.545039    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 22:16:59.545499 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.545108    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dvgzf\""
Sep 18 22:16:59.545499 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.545377    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 22:16:59.546815 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.546774    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.547520 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547443    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-cni-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.547520 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-etc-selinux\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.547520 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvlq\" (UniqueName: \"kubernetes.io/projected/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-kube-api-access-7lvlq\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.547727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31f10d1c-f78c-420a-bc96-55f7f3180896-iptables-alerter-script\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.547727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547558    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-registration-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.547727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-cni-multus\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.547727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-888jc\" (UniqueName: \"kubernetes.io/projected/31f10d1c-f78c-420a-bc96-55f7f3180896-kube-api-access-888jc\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.547727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-conf-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.547727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547647    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/13172c68-3ec9-4212-bee0-cff073c79343-multus-daemon-config\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.547669    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-multus-certs\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548528    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-cnibin\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548575    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548607    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-sys-fs\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-hostroot\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548720    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-k8s-cni-cncf-io\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31f10d1c-f78c-420a-bc96-55f7f3180896-host-slash\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548775    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-socket-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-netns\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-cni-bin\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-etc-kubernetes\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkdf2\" (UniqueName: \"kubernetes.io/projected/13172c68-3ec9-4212-bee0-cff073c79343-kube-api-access-vkdf2\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.548933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-device-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.549205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.549062    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f9cpv\""
Sep 18 22:16:59.549972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.549703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-kubelet\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.549754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-system-cni-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.549790    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13172c68-3ec9-4212-bee0-cff073c79343-cni-binary-copy\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.549820    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-socket-dir-parent\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.549972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.549847    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:16:59.550589 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.550568    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 22:16:59.550772 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.550752    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.551156 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.551126    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 22:16:59.551463 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.551446    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.551672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.551648    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 22:16:59.552070 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.552051    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.552334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.552315    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbrk8\""
Sep 18 22:16:59.552555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.552538    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 22:16:59.553411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.552766    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 22:16:59.554334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.554304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.554433 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.554371    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:16:59.556511 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.556489    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.558952 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.558931    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.559433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.559414    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 22:16:59.559790 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.559591    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4rksw\""
Sep 18 22:16:59.559879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.559799    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 22:16:59.560289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.560144    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 22:16:59.561260 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.561240    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:16:59.561368 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.561317    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:16:59.562591 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.562446    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 22:16:59.562897 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.562879    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 22:16:59.563127 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.563110    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4p6sp\""
Sep 18 22:16:59.599634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.599611    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rr87v"
Sep 18 22:16:59.612848 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.612824    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rr87v"
Sep 18 22:16:59.637439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.637400    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 22:16:59.650656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650627    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cni-binary-copy\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.650796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf8vc\" (UniqueName: \"kubernetes.io/projected/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-kube-api-access-gf8vc\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.650796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-socket-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.650796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysctl-conf\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.650796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-etc-kubernetes\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.650796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650786    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-etc-kubernetes\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkdf2\" (UniqueName: \"kubernetes.io/projected/13172c68-3ec9-4212-bee0-cff073c79343-kube-api-access-vkdf2\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650857    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c5g4\" (UniqueName: \"kubernetes.io/projected/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-kube-api-access-9c5g4\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650860    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-socket-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-run-netns\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650903    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-etc-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-ovn\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.650987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-env-overrides\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-kubelet\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651034    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-slash\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13172c68-3ec9-4212-bee0-cff073c79343-cni-binary-copy\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-socket-dir-parent\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:16:59.651138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651138    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-node-log\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-run-ovn-kubernetes\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlq\" (UniqueName: \"kubernetes.io/projected/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-kube-api-access-7lvlq\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651308    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31f10d1c-f78c-420a-bc96-55f7f3180896-iptables-alerter-script\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-kubelet\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-socket-dir-parent\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysconfig\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651478    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-systemd\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651498    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-cni-bin\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-ovnkube-config\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a26f907f-1903-4bf9-b515-62df85b2b6f7-kubelet-config\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651559    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651583    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-cni-multus\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-var-lib-kubelet\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651623    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-tuned\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4g72\" (UniqueName: \"kubernetes.io/projected/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-kube-api-access-d4g72\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.651838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-tmp-dir\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-var-lib-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-conf-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651781    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-cnibin\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651829    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8672980a-1096-46f2-a1ba-d5d1c768e247-serviceca\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31f10d1c-f78c-420a-bc96-55f7f3180896-iptables-alerter-script\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-k8s-cni-cncf-io\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cdacda15-9e2c-4c1e-a142-e5e49b023f95-konnectivity-ca\") pod \"konnectivity-agent-jst6w\" (UID: \"cdacda15-9e2c-4c1e-a142-e5e49b023f95\") " pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651927    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-systemd-units\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651951    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cnibin\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651952    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13172c68-3ec9-4212-bee0-cff073c79343-cni-binary-copy\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-cni-multus\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.651997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-tmp\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-conf-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.652681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-hosts-file\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652077    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-k8s-cni-cncf-io\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652099    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-kubelet\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-log-socket\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-cnibin\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652188    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-netns\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-cni-bin\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652287    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-netns\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-run\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-var-lib-cni-bin\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-host\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652461    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-device-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cdacda15-9e2c-4c1e-a142-e5e49b023f95-agent-certs\") pod \"konnectivity-agent-jst6w\" (UID: \"cdacda15-9e2c-4c1e-a142-e5e49b023f95\") " pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djrjw\" (UniqueName: \"kubernetes.io/projected/2f3ceffc-4830-463b-ab1e-8d1880fade2e-kube-api-access-djrjw\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:16:59.653492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652619    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-device-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-system-cni-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652669    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-modprobe-d\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652690    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-sys\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8672980a-1096-46f2-a1ba-d5d1c768e247-host\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652776    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-system-cni-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znd94\" (UniqueName: \"kubernetes.io/projected/8672980a-1096-46f2-a1ba-d5d1c768e247-kube-api-access-znd94\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-cni-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-etc-selinux\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652855    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysctl-d\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84d38006-4cd1-4768-837a-545851d17b8f-ovn-node-metrics-cert\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-ovnkube-script-lib\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652926    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-etc-selinux\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-registration-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652970    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-multus-cni-dir\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.652977    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-888jc\" (UniqueName: \"kubernetes.io/projected/31f10d1c-f78c-420a-bc96-55f7f3180896-kube-api-access-888jc\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-kubernetes\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.654478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-cni-netd\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31f10d1c-f78c-420a-bc96-55f7f3180896-host-slash\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-lib-modules\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653176    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31f10d1c-f78c-420a-bc96-55f7f3180896-host-slash\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-systemd\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/13172c68-3ec9-4212-bee0-cff073c79343-multus-daemon-config\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653251    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-multus-certs\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653257    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-registration-dir\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-sys-fs\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653363    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-host-run-multus-certs\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-hostroot\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-sys-fs\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdvll\" (UniqueName: \"kubernetes.io/projected/84d38006-4cd1-4768-837a-545851d17b8f-kube-api-access-gdvll\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653518    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a26f907f-1903-4bf9-b515-62df85b2b6f7-dbus\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/13172c68-3ec9-4212-bee0-cff073c79343-hostroot\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.655375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-system-cni-dir\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.656264 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.653708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/13172c68-3ec9-4212-bee0-cff073c79343-multus-daemon-config\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.658115 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.658061    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:16:59.658115 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.658085    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:16:59.658115 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.658099    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:16:59.658328 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.658182    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:00.158144671 +0000 UTC m=+4.518661512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:16:59.658578 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.658558    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 22:16:59.663255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.663236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-888jc\" (UniqueName: \"kubernetes.io/projected/31f10d1c-f78c-420a-bc96-55f7f3180896-kube-api-access-888jc\") pod \"iptables-alerter-4nvgh\" (UID: \"31f10d1c-f78c-420a-bc96-55f7f3180896\") " pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.663347 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.663276    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvlq\" (UniqueName: \"kubernetes.io/projected/0e4dd806-d4bb-47ec-aff3-c548d4f8a00c-kube-api-access-7lvlq\") pod \"aws-ebs-csi-driver-node-95cc6\" (UID: \"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.663347 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.663276    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkdf2\" (UniqueName: \"kubernetes.io/projected/13172c68-3ec9-4212-bee0-cff073c79343-kube-api-access-vkdf2\") pod \"multus-jwgtn\" (UID: \"13172c68-3ec9-4212-bee0-cff073c79343\") " pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.682618 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.682546    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal" event={"ID":"f394ebf0066db702356187a5a575a284","Type":"ContainerStarted","Data":"f6902ce8e7b2163ab8f0ddb2575faf5d47fbd8103df23ffa9be0b81440d733e0"}
Sep 18 22:16:59.684317 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.684290    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" event={"ID":"479909f6ffdb4c7950f561521538164e","Type":"ContainerStarted","Data":"0ffd30029c346b4bf1601eb71b6bd752b800ab78b3331bdeb4c4fb42728ff9f1"}
Sep 18 22:16:59.753822 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753795    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysctl-conf\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.753822 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c5g4\" (UniqueName: \"kubernetes.io/projected/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-kube-api-access-9c5g4\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-run-netns\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753895    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-run-netns\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-etc-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753975    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-ovn\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753976    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-etc-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.753993    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysctl-conf\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-env-overrides\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-ovn\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-slash\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-node-log\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-run-ovn-kubernetes\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754169    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-slash\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-run-ovn-kubernetes\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754208    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-node-log\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysconfig\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754273    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-systemd\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysconfig\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-cni-bin\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-ovnkube-config\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-systemd\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a26f907f-1903-4bf9-b515-62df85b2b6f7-kubelet-config\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-cni-bin\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754402    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a26f907f-1903-4bf9-b515-62df85b2b6f7-kubelet-config\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754440    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-var-lib-kubelet\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.754497 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.754465    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.754547    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:00.254534433 +0000 UTC m=+4.615051268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754550    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-var-lib-kubelet\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-tuned\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754566    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-env-overrides\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4g72\" (UniqueName: \"kubernetes.io/projected/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-kube-api-access-d4g72\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754617    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-tmp-dir\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754641    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-var-lib-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754689    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8672980a-1096-46f2-a1ba-d5d1c768e247-serviceca\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754714    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cdacda15-9e2c-4c1e-a142-e5e49b023f95-konnectivity-ca\") pod \"konnectivity-agent-jst6w\" (UID: \"cdacda15-9e2c-4c1e-a142-e5e49b023f95\") " pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754724    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-var-lib-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-systemd-units\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-systemd-units\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754834    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-ovnkube-config\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cnibin\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754872    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-openvswitch\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.755438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-tmp\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cnibin\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.754950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-hosts-file\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-hosts-file\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755013    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-tmp-dir\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-kubelet\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755063    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-log-socket\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-run\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-host\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755126    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755157    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-log-socket\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755168    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8672980a-1096-46f2-a1ba-d5d1c768e247-serviceca\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cdacda15-9e2c-4c1e-a142-e5e49b023f95-agent-certs\") pod \"konnectivity-agent-jst6w\" (UID: \"cdacda15-9e2c-4c1e-a142-e5e49b023f95\") " pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djrjw\" (UniqueName: \"kubernetes.io/projected/2f3ceffc-4830-463b-ab1e-8d1880fade2e-kube-api-access-djrjw\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755233    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-host\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755247    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-run\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.756369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755203    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-kubelet\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-modprobe-d\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755287    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-sys\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755308    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8672980a-1096-46f2-a1ba-d5d1c768e247-host\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znd94\" (UniqueName: \"kubernetes.io/projected/8672980a-1096-46f2-a1ba-d5d1c768e247-kube-api-access-znd94\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755339    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cdacda15-9e2c-4c1e-a142-e5e49b023f95-konnectivity-ca\") pod \"konnectivity-agent-jst6w\" (UID: \"cdacda15-9e2c-4c1e-a142-e5e49b023f95\") " pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysctl-d\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84d38006-4cd1-4768-837a-545851d17b8f-ovn-node-metrics-cert\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-ovnkube-script-lib\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755448    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-kubernetes\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755450    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-cni-netd\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755513    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-kubernetes\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755529    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755558    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-lib-modules\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755577    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755458    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-sys\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.757163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-systemd\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-run-systemd\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755667    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdvll\" (UniqueName: \"kubernetes.io/projected/84d38006-4cd1-4768-837a-545851d17b8f-kube-api-access-gdvll\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755720    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a26f907f-1903-4bf9-b515-62df85b2b6f7-dbus\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755744    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-system-cni-dir\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8672980a-1096-46f2-a1ba-d5d1c768e247-host\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755765    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-sysctl-d\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cni-binary-copy\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf8vc\" (UniqueName: \"kubernetes.io/projected/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-kube-api-access-gf8vc\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755901    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a26f907f-1903-4bf9-b515-62df85b2b6f7-dbus\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.755775    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-system-cni-dir\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.755982    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:16:59.756051    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:17:00.256035741 +0000 UTC m=+4.616552580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.756076    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84d38006-4cd1-4768-837a-545851d17b8f-ovnkube-script-lib\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.756131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84d38006-4cd1-4768-837a-545851d17b8f-host-cni-netd\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.756157    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-lib-modules\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.756303    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-modprobe-d\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.756467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-cni-binary-copy\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.756525    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.757525    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-tmp\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.757788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-etc-tuned\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.758500    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84d38006-4cd1-4768-837a-545851d17b8f-ovn-node-metrics-cert\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.758924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.758537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cdacda15-9e2c-4c1e-a142-e5e49b023f95-agent-certs\") pod \"konnectivity-agent-jst6w\" (UID: \"cdacda15-9e2c-4c1e-a142-e5e49b023f95\") " pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.762218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.762186    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c5g4\" (UniqueName: \"kubernetes.io/projected/ee7ea442-4aa4-4aea-a9f0-3639acc94dc9-kube-api-access-9c5g4\") pod \"tuned-5p2xm\" (UID: \"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9\") " pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.766085 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.766023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4g72\" (UniqueName: \"kubernetes.io/projected/b39dd412-80b2-4d98-b06a-8c0f1b86a59b-kube-api-access-d4g72\") pod \"node-resolver-r28kg\" (UID: \"b39dd412-80b2-4d98-b06a-8c0f1b86a59b\") " pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.766266 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.766247    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djrjw\" (UniqueName: \"kubernetes.io/projected/2f3ceffc-4830-463b-ab1e-8d1880fade2e-kube-api-access-djrjw\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:16:59.766441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.766359    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdvll\" (UniqueName: \"kubernetes.io/projected/84d38006-4cd1-4768-837a-545851d17b8f-kube-api-access-gdvll\") pod \"ovnkube-node-km2gr\" (UID: \"84d38006-4cd1-4768-837a-545851d17b8f\") " pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.766895 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.766875    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znd94\" (UniqueName: \"kubernetes.io/projected/8672980a-1096-46f2-a1ba-d5d1c768e247-kube-api-access-znd94\") pod \"node-ca-gdk85\" (UID: \"8672980a-1096-46f2-a1ba-d5d1c768e247\") " pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.767108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.767056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf8vc\" (UniqueName: \"kubernetes.io/projected/0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9-kube-api-access-gf8vc\") pod \"multus-additional-cni-plugins-qjr2x\" (UID: \"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9\") " pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:16:59.843420 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.843329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jwgtn"
Sep 18 22:16:59.866962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.866935    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4nvgh"
Sep 18 22:16:59.870665 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.870647    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6"
Sep 18 22:16:59.876308 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.876290    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5p2xm"
Sep 18 22:16:59.884807 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.884789    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:16:59.890453 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.890428    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r28kg"
Sep 18 22:16:59.898095 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.898075    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:16:59.905635 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.905617    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gdk85"
Sep 18 22:16:59.913136 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:16:59.913118    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qjr2x"
Sep 18 22:17:00.064843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.064813    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:17:00.159543 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.159511    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:00.159731 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.159649    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:17:00.159731 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.159666    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:17:00.159731 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.159676    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:00.159838 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.159732    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:01.159713596 +0000 UTC m=+5.520230435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:00.259833 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.259801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:00.259981 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.259846    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:00.259981 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.259969    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:00.260100 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.259985    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:00.260100 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.260046    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:01.260027475 +0000 UTC m=+5.620544320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:00.260100 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.260067    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:17:01.260055727 +0000 UTC m=+5.620572565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:00.616624 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.616527    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 22:11:59 +0000 UTC" deadline="2028-04-18 05:25:41.726627189 +0000 UTC"
Sep 18 22:17:00.616624 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.616552    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13855h8m41.110078083s"
Sep 18 22:17:00.633032 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:00.633005    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8bb6e2_d39b_4b10_b2dd_28c84e42c2e9.slice/crio-086cb2f8432ebf1c4de326cf4b458dd15f8df6507a6e88185814c11c9e3fdefa WatchSource:0}: Error finding container 086cb2f8432ebf1c4de326cf4b458dd15f8df6507a6e88185814c11c9e3fdefa: Status 404 returned error can't find the container with id 086cb2f8432ebf1c4de326cf4b458dd15f8df6507a6e88185814c11c9e3fdefa
Sep 18 22:17:00.634771 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:00.634736    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d38006_4cd1_4768_837a_545851d17b8f.slice/crio-4126fedf98f4fc697045e25e649bbe79a73bff10dc5bc06abc0a97648f78947b WatchSource:0}: Error finding container 4126fedf98f4fc697045e25e649bbe79a73bff10dc5bc06abc0a97648f78947b: Status 404 returned error can't find the container with id 4126fedf98f4fc697045e25e649bbe79a73bff10dc5bc06abc0a97648f78947b
Sep 18 22:17:00.636300 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:00.636273    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7ea442_4aa4_4aea_a9f0_3639acc94dc9.slice/crio-4f8f3cad2e5b48388b0ad7668f9ff7c4c8d65d21ad94723cc6c30d292f92b617 WatchSource:0}: Error finding container 4f8f3cad2e5b48388b0ad7668f9ff7c4c8d65d21ad94723cc6c30d292f92b617: Status 404 returned error can't find the container with id 4f8f3cad2e5b48388b0ad7668f9ff7c4c8d65d21ad94723cc6c30d292f92b617
Sep 18 22:17:00.658674 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:00.658653    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8672980a_1096_46f2_a1ba_d5d1c768e247.slice/crio-d6f47407065b4c0962391caa90d6183800b594521b5b1b959c0cfa6714327262 WatchSource:0}: Error finding container d6f47407065b4c0962391caa90d6183800b594521b5b1b959c0cfa6714327262: Status 404 returned error can't find the container with id d6f47407065b4c0962391caa90d6183800b594521b5b1b959c0cfa6714327262
Sep 18 22:17:00.665632 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:00.665613    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13172c68_3ec9_4212_bee0_cff073c79343.slice/crio-402347f3c26179702e5dd574759b489fe59b1b78074d9766e7079815a0cc9ede WatchSource:0}: Error finding container 402347f3c26179702e5dd574759b489fe59b1b78074d9766e7079815a0cc9ede: Status 404 returned error can't find the container with id 402347f3c26179702e5dd574759b489fe59b1b78074d9766e7079815a0cc9ede
Sep 18 22:17:00.676344 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.676322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:00.676454 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:00.676432    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:00.689920 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.689885    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6" event={"ID":"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c","Type":"ContainerStarted","Data":"ab8c6fbb9fde4a1079a2e1aa9c55af5fedc9f087487280c8bf6f7096578e078f"}
Sep 18 22:17:00.692326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.692259    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gdk85" event={"ID":"8672980a-1096-46f2-a1ba-d5d1c768e247","Type":"ContainerStarted","Data":"d6f47407065b4c0962391caa90d6183800b594521b5b1b959c0cfa6714327262"}
Sep 18 22:17:00.694121 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.694100    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r28kg" event={"ID":"b39dd412-80b2-4d98-b06a-8c0f1b86a59b","Type":"ContainerStarted","Data":"cc23163265675ec01922ca08e4c678e914bc98e7745bf4bf18d1ed8e122b0c3e"}
Sep 18 22:17:00.696218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.696172    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5p2xm" event={"ID":"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9","Type":"ContainerStarted","Data":"4f8f3cad2e5b48388b0ad7668f9ff7c4c8d65d21ad94723cc6c30d292f92b617"}
Sep 18 22:17:00.699482 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.698142    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"4126fedf98f4fc697045e25e649bbe79a73bff10dc5bc06abc0a97648f78947b"}
Sep 18 22:17:00.699928 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.699905    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerStarted","Data":"086cb2f8432ebf1c4de326cf4b458dd15f8df6507a6e88185814c11c9e3fdefa"}
Sep 18 22:17:00.701090 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.701065    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jwgtn" event={"ID":"13172c68-3ec9-4212-bee0-cff073c79343","Type":"ContainerStarted","Data":"402347f3c26179702e5dd574759b489fe59b1b78074d9766e7079815a0cc9ede"}
Sep 18 22:17:00.702145 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.702124    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4nvgh" event={"ID":"31f10d1c-f78c-420a-bc96-55f7f3180896","Type":"ContainerStarted","Data":"7cba1655c9b2c00522bd92b5c019db29820e01b14606531316c199b2a56b2058"}
Sep 18 22:17:00.703193 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:00.703176    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jst6w" event={"ID":"cdacda15-9e2c-4c1e-a142-e5e49b023f95","Type":"ContainerStarted","Data":"e4d7b3276fb741f891abcba82ce0e60598bd384a4386d7750ee49513aecf52bf"}
Sep 18 22:17:01.167222 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.166600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:01.167222 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.166801    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:17:01.167222 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.166819    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:17:01.167222 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.166832    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:01.167222 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.166891    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:03.166872129 +0000 UTC m=+7.527388967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:01.268021 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.267237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:01.268021 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.267301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:01.268021 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.267462    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:01.268021 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.267526    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:17:03.267507349 +0000 UTC m=+7.628024190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:01.268021 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.267916    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:01.268021 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.267977    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:03.267961355 +0000 UTC m=+7.628478196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:01.617601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.617447    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 22:11:59 +0000 UTC" deadline="2028-02-13 07:46:16.686810005 +0000 UTC"
Sep 18 22:17:01.617601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.617508    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12297h29m15.069305832s"
Sep 18 22:17:01.676554 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.676524    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:01.676773 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.676690    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:01.677175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.677156    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:01.677284 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:01.677263    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:01.711980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.711942    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal" event={"ID":"f394ebf0066db702356187a5a575a284","Type":"ContainerStarted","Data":"0eefb7d6ea7a3d791ca5890784504b7efb1d6c400a8dc50954561f4088c2dbe2"}
Sep 18 22:17:01.715718 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.715688    2583 generic.go:356] "Generic (PLEG): container finished" podID="479909f6ffdb4c7950f561521538164e" containerID="923bc6a5caf7267265e075062778611c4b0326cd1c509d60782852cbb6757427" exitCode=0
Sep 18 22:17:01.715841 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.715736    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" event={"ID":"479909f6ffdb4c7950f561521538164e","Type":"ContainerDied","Data":"923bc6a5caf7267265e075062778611c4b0326cd1c509d60782852cbb6757427"}
Sep 18 22:17:01.744124 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:01.744065    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal" podStartSLOduration=3.744046924 podStartE2EDuration="3.744046924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:01.727239107 +0000 UTC m=+6.087755965" watchObservedRunningTime="2026-09-18 22:17:01.744046924 +0000 UTC m=+6.104563782"
Sep 18 22:17:02.677232 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:02.677140    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:02.677690 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:02.677319    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:02.751506 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:02.751441    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" event={"ID":"479909f6ffdb4c7950f561521538164e","Type":"ContainerStarted","Data":"eae3b86fd2c206b3c1fb3133394c4198e5a850681cb660802b9634cc8c811333"}
Sep 18 22:17:02.770774 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:02.770719    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" podStartSLOduration=4.770694821 podStartE2EDuration="4.770694821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:02.769511672 +0000 UTC m=+7.130028532" watchObservedRunningTime="2026-09-18 22:17:02.770694821 +0000 UTC m=+7.131211678"
Sep 18 22:17:03.191466 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:03.190957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:03.191466 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.191107    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:17:03.191466 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.191122    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:17:03.191466 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.191131    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:03.191466 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.191176    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:07.191163954 +0000 UTC m=+11.551680794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:03.292312 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:03.292275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:03.292502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:03.292365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:03.292583 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.292513    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:03.292583 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.292577    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:07.292558543 +0000 UTC m=+11.653075383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:03.293008 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.292987    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:03.293093 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.293048    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:17:07.293029685 +0000 UTC m=+11.653546522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:03.678190 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:03.677515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:03.678190 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.677649    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:03.678190 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:03.678005    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:03.678190 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:03.678117    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:04.676978 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:04.676941    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:04.677161 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:04.677085    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:05.677402 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:05.677359    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:05.677909 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:05.677419    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:05.677909 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:05.677523    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:05.677909 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:05.677642    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:06.677264 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:06.676742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:06.677264 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:06.676889    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:07.227466 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:07.227421    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:07.227718 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.227651    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:17:07.227718 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.227671    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:17:07.227718 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.227685    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:07.227866 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.227751    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:15.227731509 +0000 UTC m=+19.588248347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:07.328621 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:07.328584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:07.328808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:07.328646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:07.328808 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.328771    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:07.328808 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.328784    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:07.328956 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.328836    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:17:15.328817143 +0000 UTC m=+19.689333982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:07.328956 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.328855    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:15.328845216 +0000 UTC m=+19.689362054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:07.677874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:07.677840    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:07.678316 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.677947    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:07.678316 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:07.677989    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:07.678316 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:07.678128    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:08.677065 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:08.676804    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:08.677252 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:08.677191    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:09.676594 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:09.676559    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:09.676594 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:09.676597    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:09.677107 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:09.676684    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:09.677107 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:09.676794    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:10.676912 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:10.676870    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:10.677361 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:10.676997    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:11.676552 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:11.676521    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:11.676864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:11.676521    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:11.676864 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:11.676666    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:11.676864 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:11.676710    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:12.676686 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:12.676655    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:12.676853 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:12.676759    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:13.676916 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:13.676845    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:13.677440 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:13.676853    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:13.677440 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:13.676998    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:13.677440 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:13.677071    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:14.677134 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:14.677100    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:14.677596 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:14.677220    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:15.289149 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:15.289113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:15.289316 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.289301    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:17:15.289368 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.289324    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:17:15.289368 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.289334    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:15.289508 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.289408    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:31.28937606 +0000 UTC m=+35.649892918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:15.389715 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:15.389678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:15.389715 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:15.389721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:15.389928 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.389830    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:15.389928 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.389888    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:17:31.389875179 +0000 UTC m=+35.750392027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:15.389928 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.389830    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:15.390058 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.389958    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:31.389946475 +0000 UTC m=+35.750463323 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:15.677239 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:15.677203    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:15.677814 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.677340    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:15.677814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:15.677400    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:15.677814 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:15.677538    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:16.677099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:16.677068    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:16.677272 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:16.677178    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:17.676941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:17.676910    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:17.677107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:17.676993    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:17.677107 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:17.677017    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:17.677107 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:17.677086    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:18.677156 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.677138    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:18.677643 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:18.677231    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:18.786517 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.786478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gdk85" event={"ID":"8672980a-1096-46f2-a1ba-d5d1c768e247","Type":"ContainerStarted","Data":"e2f5949ee0f647c4c803e5eb2d228104d4de6b9a6a1184aab0e0ce029580722f"}
Sep 18 22:17:18.788245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.788220    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r28kg" event={"ID":"b39dd412-80b2-4d98-b06a-8c0f1b86a59b","Type":"ContainerStarted","Data":"48144e8b981b0a3545b622377ffc6d93a3ef62c1cab9a9d85295037ae2a3d4e4"}
Sep 18 22:17:18.790565 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.790536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5p2xm" event={"ID":"ee7ea442-4aa4-4aea-a9f0-3639acc94dc9","Type":"ContainerStarted","Data":"75c0e014f87393a4e2dd3535253614a87f1e612a65e36c3e22515fcb8843b1a3"}
Sep 18 22:17:18.794476 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.794450    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"0835f842aa157812ea05ac8285b44e15fc8a73d24cbd8914ed9eb01cb14bd244"}
Sep 18 22:17:18.794575 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.794529    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"872887f38e45121fddfd0bfd72c2d5b68fd119b92c5a855f14e900b5be964504"}
Sep 18 22:17:18.794639 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.794580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"193f35f0bf794cc2a07ccbeb6cd2f6ec43893dc943083fe6a0d3140e0f000ffd"}
Sep 18 22:17:18.794639 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.794625    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"be5d7d15ccf7bee178f99095b5aa9387b16ae80ae960ac8f10c2df97156036d1"}
Sep 18 22:17:18.794734 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.794655    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"8a4bc8e7148e18f45c689d77e4dfd71d8ccf9822d9a98c03926c37f0a92eb028"}
Sep 18 22:17:18.794734 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.794695    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"1dff8f9d864d0fb6906e2abee6f2289ca017f03068e56f8389c9c2dd3d83fb67"}
Sep 18 22:17:18.796433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.796406    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9" containerID="205a858900ac0eeadcc373cbe102571dd970b152850402040dc6085c17f22dfb" exitCode=0
Sep 18 22:17:18.796537 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.796482    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerDied","Data":"205a858900ac0eeadcc373cbe102571dd970b152850402040dc6085c17f22dfb"}
Sep 18 22:17:18.798751 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.798727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jwgtn" event={"ID":"13172c68-3ec9-4212-bee0-cff073c79343","Type":"ContainerStarted","Data":"621f0ea4a19eed095cf621cc312247bec54ccb66bc452b37541d7c1a4aff98c7"}
Sep 18 22:17:18.800833 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.800640    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jst6w" event={"ID":"cdacda15-9e2c-4c1e-a142-e5e49b023f95","Type":"ContainerStarted","Data":"9f525791addee3f29d57ce79836edd9776f8523983b81535bf3dfc09d85fe9c5"}
Sep 18 22:17:18.802482 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.802452    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6" event={"ID":"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c","Type":"ContainerStarted","Data":"a118056d4243388dda935e227f78a77942f41f2df022f36f98b5992cb9734f01"}
Sep 18 22:17:18.802950 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.802909    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gdk85" podStartSLOduration=4.356672116 podStartE2EDuration="21.802894003s" totalImagesPullingTime="17.446221887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:18.802497218 +0000 UTC m=+23.163014078" watchObservedRunningTime="2026-09-18 22:17:18.802894003 +0000 UTC m=+23.163410861"
Sep 18 22:17:18.825761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.825711    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5p2xm" podStartSLOduration=4.372353211 podStartE2EDuration="21.82569275s" totalImagesPullingTime="17.453339539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:18.825125012 +0000 UTC m=+23.185641883" watchObservedRunningTime="2026-09-18 22:17:18.82569275 +0000 UTC m=+23.186209607"
Sep 18 22:17:18.842225 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.842135    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jst6w" podStartSLOduration=8.393189726 podStartE2EDuration="21.842118605s" totalImagesPullingTime="13.448928879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:18.841524838 +0000 UTC m=+23.202041695" watchObservedRunningTime="2026-09-18 22:17:18.842118605 +0000 UTC m=+23.202635464"
Sep 18 22:17:18.857809 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.857763    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r28kg" podStartSLOduration=8.408818942 podStartE2EDuration="21.857748788s" totalImagesPullingTime="13.448929846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:18.857150279 +0000 UTC m=+23.217667138" watchObservedRunningTime="2026-09-18 22:17:18.857748788 +0000 UTC m=+23.218265644"
Sep 18 22:17:18.898431 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:18.898359    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jwgtn" podStartSLOduration=4.447848607 podStartE2EDuration="21.898345214s" totalImagesPullingTime="17.450496607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:18.897926459 +0000 UTC m=+23.258443315" watchObservedRunningTime="2026-09-18 22:17:18.898345214 +0000 UTC m=+23.258862068"
Sep 18 22:17:19.218380 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.218359    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 22:17:19.646125 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.646017    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T22:17:19.218376787Z","UUID":"7d8b7453-6d47-4d43-b2d0-9127ae973deb","Handler":null,"Name":"","Endpoint":""}
Sep 18 22:17:19.647766 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.647734    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 22:17:19.647766 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.647768    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 22:17:19.676608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.676416    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:19.676861 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.676440    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:19.676861 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:19.676708    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:19.676861 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:19.676768    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:19.807288 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.806868    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4nvgh" event={"ID":"31f10d1c-f78c-420a-bc96-55f7f3180896","Type":"ContainerStarted","Data":"2c3bfce500eec381d267ab61b3bfe62192b3457320502bcce401f63d4ed64ad1"}
Sep 18 22:17:19.809316 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.809285    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6" event={"ID":"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c","Type":"ContainerStarted","Data":"1d0a487e6fb56acfc8353f540765a60dfa5a44f967559df3849a20921d5198e0"}
Sep 18 22:17:19.823119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:19.823075    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4nvgh" podStartSLOduration=5.410134186 podStartE2EDuration="22.823061362s" totalImagesPullingTime="17.412927176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:19.822959686 +0000 UTC m=+24.183476546" watchObservedRunningTime="2026-09-18 22:17:19.823061362 +0000 UTC m=+24.183578219"
Sep 18 22:17:20.677248 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:20.677211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:20.677424 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:20.677341    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:20.814634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:20.814598    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6" event={"ID":"0e4dd806-d4bb-47ec-aff3-c548d4f8a00c","Type":"ContainerStarted","Data":"bed3d99fac09f75f8cdd1d2139fd94b4d2a82b3d36df8d430cf14f4f2d68851f"}
Sep 18 22:17:20.819200 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:20.819171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"965b1d14395dc4ecc875e8b0c1bcc377816f4f154e8aee647bf579d08c6b462f"}
Sep 18 22:17:21.676370 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:21.676339    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:21.676597 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:21.676476    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:21.676597 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:21.676522    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:21.676705 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:21.676624    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:22.676810 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:22.676782    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:22.677174 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:22.676901    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:22.977572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:22.977503    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:17:22.978496 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:22.978479    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:17:22.993300 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:22.993264    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-95cc6" podStartSLOduration=6.994169444 podStartE2EDuration="25.993251114s" totalImagesPullingTime="18.99908167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:20.832984102 +0000 UTC m=+25.193500969" watchObservedRunningTime="2026-09-18 22:17:22.993251114 +0000 UTC m=+27.353768027"
Sep 18 22:17:23.677108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.676936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:23.677460 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.676947    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:23.677460 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:23.677173    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:23.677460 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:23.677299    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:23.829298 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.829260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" event={"ID":"84d38006-4cd1-4768-837a-545851d17b8f","Type":"ContainerStarted","Data":"6f1da753432a6891845efce658e435a49f0ec11d89a28be93f696eaca49b6188"}
Sep 18 22:17:23.829522 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.829506    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:17:23.831436 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.831413    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9" containerID="34710f7d705772c708b3e29df2261a3e579aa979c9b6f13c2c8a3214d3e2987c" exitCode=0
Sep 18 22:17:23.831543 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.831500    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerDied","Data":"34710f7d705772c708b3e29df2261a3e579aa979c9b6f13c2c8a3214d3e2987c"}
Sep 18 22:17:23.831691 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.831678    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:17:23.832208 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.832194    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jst6w"
Sep 18 22:17:23.845008 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.844980    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:17:23.859043 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:23.859000    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr" podStartSLOduration=9.367922208 podStartE2EDuration="26.858987201s" totalImagesPullingTime="17.491064993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:23.858683378 +0000 UTC m=+28.219200269" watchObservedRunningTime="2026-09-18 22:17:23.858987201 +0000 UTC m=+28.219504057"
Sep 18 22:17:24.676882 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:24.676838    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:24.677090 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:24.676941    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:24.833314 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:24.833287    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:17:24.833911 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:24.833373    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:17:24.847970 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:24.847949    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:17:25.012659 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.012629    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lwvjn"]
Sep 18 22:17:25.012777 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.012763    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:25.012867 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:25.012847    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:25.015286 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.015254    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xzzt2"]
Sep 18 22:17:25.015522 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.015365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:25.015522 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:25.015470    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:25.015834 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.015814    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4t6gh"]
Sep 18 22:17:25.015926 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.015913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:25.016016 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:25.015997    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:25.837732 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.837701    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9" containerID="eca80b2814837ded4783e6129da92959d4dbbc743f15bee3ed80fc5e445a61da" exitCode=0
Sep 18 22:17:25.838219 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:25.837797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerDied","Data":"eca80b2814837ded4783e6129da92959d4dbbc743f15bee3ed80fc5e445a61da"}
Sep 18 22:17:26.677302 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:26.677269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:26.677523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:26.677269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:26.677523 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:26.677421    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:26.677523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:26.677269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:26.677523 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:26.677466    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:26.677523 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:26.677504    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:27.845087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:27.844876    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9" containerID="465307f5205ef2a0e71c77d63d06d2a067c39fec4276543da15ea348a31a1609" exitCode=0
Sep 18 22:17:27.845087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:27.844921    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerDied","Data":"465307f5205ef2a0e71c77d63d06d2a067c39fec4276543da15ea348a31a1609"}
Sep 18 22:17:28.676801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:28.676758    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:28.676801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:28.676789    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:28.677028 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:28.676772    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:28.677028 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:28.676903    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:28.677028 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:28.676965    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:28.677197 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:28.677061    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:30.677324 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:30.677288    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:30.677867 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:30.677304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:30.677867 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:30.677445    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xzzt2" podUID="2f3ceffc-4830-463b-ab1e-8d1880fade2e"
Sep 18 22:17:30.677867 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:30.677304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:30.677867 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:30.677488    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4t6gh" podUID="a26f907f-1903-4bf9-b515-62df85b2b6f7"
Sep 18 22:17:30.677867 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:30.677591    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lwvjn" podUID="6c7417ee-d304-4d98-9ade-8db75a074ea9"
Sep 18 22:17:31.303170 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:31.303088    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:31.303340 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.303231    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 22:17:31.303340 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.303246    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 22:17:31.303340 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.303255    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vxdgf for pod openshift-network-diagnostics/network-check-target-lwvjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:31.303340 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.303320    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf podName:6c7417ee-d304-4d98-9ade-8db75a074ea9 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:03.303299568 +0000 UTC m=+67.663816436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vxdgf" (UniqueName: "kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf") pod "network-check-target-lwvjn" (UID: "6c7417ee-d304-4d98-9ade-8db75a074ea9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 22:17:31.403718 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:31.403675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:31.403889 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:31.403739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:31.403889 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.403838    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:31.403889 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.403843    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:31.404000 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.403900    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret podName:a26f907f-1903-4bf9-b515-62df85b2b6f7 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:03.403881812 +0000 UTC m=+67.764398648 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret") pod "global-pull-secret-syncer-4t6gh" (UID: "a26f907f-1903-4bf9-b515-62df85b2b6f7") : object "kube-system"/"original-pull-secret" not registered
Sep 18 22:17:31.404000 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:31.403914    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:18:03.403907661 +0000 UTC m=+67.764424495 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 22:17:32.104701 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.104629    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeReady"
Sep 18 22:17:32.105044 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.104748    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 22:17:32.194755 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.194730    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sqsj6"]
Sep 18 22:17:32.221042 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.221015    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6hg5t"]
Sep 18 22:17:32.221168 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.221139    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.223691 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.223668    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 22:17:32.224267 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.224232    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 22:17:32.224624 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.224608    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gtntj\""
Sep 18 22:17:32.241139 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.241118    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hg5t"]
Sep 18 22:17:32.241246 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.241158    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sqsj6"]
Sep 18 22:17:32.241306 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.241241    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.244222 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.244196    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 22:17:32.244304 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.244242    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lv824\""
Sep 18 22:17:32.244642 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.244613    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 22:17:32.244974 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.244957    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 22:17:32.310549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.310502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhrcd\" (UniqueName: \"kubernetes.io/projected/c843f1a2-c11e-4e96-aa81-998169823479-kube-api-access-xhrcd\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.310549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.310541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.310549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.310561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c843f1a2-c11e-4e96-aa81-998169823479-tmp-dir\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.310855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.310624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjjm\" (UniqueName: \"kubernetes.io/projected/309bb642-370f-4c18-896e-844cb37b7672-kube-api-access-bbjjm\") pod \"ingress-canary-6hg5t\" (UID: \"309bb642-370f-4c18-896e-844cb37b7672\") " pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.310855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.310664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/309bb642-370f-4c18-896e-844cb37b7672-cert\") pod \"ingress-canary-6hg5t\" (UID: \"309bb642-370f-4c18-896e-844cb37b7672\") " pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.310855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.310735    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c843f1a2-c11e-4e96-aa81-998169823479-config-volume\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.411489 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhrcd\" (UniqueName: \"kubernetes.io/projected/c843f1a2-c11e-4e96-aa81-998169823479-kube-api-access-xhrcd\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.411489 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.411647 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c843f1a2-c11e-4e96-aa81-998169823479-tmp-dir\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.411647 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjjm\" (UniqueName: \"kubernetes.io/projected/309bb642-370f-4c18-896e-844cb37b7672-kube-api-access-bbjjm\") pod \"ingress-canary-6hg5t\" (UID: \"309bb642-370f-4c18-896e-844cb37b7672\") " pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.411647 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/309bb642-370f-4c18-896e-844cb37b7672-cert\") pod \"ingress-canary-6hg5t\" (UID: \"309bb642-370f-4c18-896e-844cb37b7672\") " pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.411647 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c843f1a2-c11e-4e96-aa81-998169823479-config-volume\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.411647 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:32.411635    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:17:32.411898 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:32.411703    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:32.911684255 +0000 UTC m=+37.272201095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:17:32.411898 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.411875    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c843f1a2-c11e-4e96-aa81-998169823479-tmp-dir\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.412065 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.412046    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c843f1a2-c11e-4e96-aa81-998169823479-config-volume\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.416050 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.416028    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/309bb642-370f-4c18-896e-844cb37b7672-cert\") pod \"ingress-canary-6hg5t\" (UID: \"309bb642-370f-4c18-896e-844cb37b7672\") " pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.422320 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.422299    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjjm\" (UniqueName: \"kubernetes.io/projected/309bb642-370f-4c18-896e-844cb37b7672-kube-api-access-bbjjm\") pod \"ingress-canary-6hg5t\" (UID: \"309bb642-370f-4c18-896e-844cb37b7672\") " pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.422459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.422367    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhrcd\" (UniqueName: \"kubernetes.io/projected/c843f1a2-c11e-4e96-aa81-998169823479-kube-api-access-xhrcd\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.551266 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.551231    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hg5t"
Sep 18 22:17:32.676820 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.676633    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:17:32.676939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.676683    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:17:32.676939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.676685    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:17:32.679810 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.679791    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 22:17:32.679810 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.679791    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 22:17:32.679810 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.679791    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 22:17:32.680066 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.680020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 22:17:32.680816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.680801    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-95pvs\""
Sep 18 22:17:32.680906 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.680844    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dq7t2\""
Sep 18 22:17:32.861763 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.861729    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9" containerID="db500431aa2c3c73f0f890dc26ef797f38aa20e1602fbbe175a23503df0666ae" exitCode=0
Sep 18 22:17:32.861924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.861783    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerDied","Data":"db500431aa2c3c73f0f890dc26ef797f38aa20e1602fbbe175a23503df0666ae"}
Sep 18 22:17:32.875059 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.875038    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hg5t"]
Sep 18 22:17:32.878896 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:32.878874    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309bb642_370f_4c18_896e_844cb37b7672.slice/crio-74b716cfaf15f1045577ffbb5afae625d28793c8c6228a5090abe25ae8b7cec6 WatchSource:0}: Error finding container 74b716cfaf15f1045577ffbb5afae625d28793c8c6228a5090abe25ae8b7cec6: Status 404 returned error can't find the container with id 74b716cfaf15f1045577ffbb5afae625d28793c8c6228a5090abe25ae8b7cec6
Sep 18 22:17:32.916804 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:32.916776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:32.916943 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:32.916921    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:17:32.917067 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:32.917005    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:33.916985259 +0000 UTC m=+38.277502111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:17:33.866119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:33.866082    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hg5t" event={"ID":"309bb642-370f-4c18-896e-844cb37b7672","Type":"ContainerStarted","Data":"74b716cfaf15f1045577ffbb5afae625d28793c8c6228a5090abe25ae8b7cec6"}
Sep 18 22:17:33.869747 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:33.869718    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9" containerID="f814e22f276b7677be255aa7ed103a6ad14cb1f89dacdc4d76cbcf2126e10e09" exitCode=0
Sep 18 22:17:33.869870 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:33.869768    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerDied","Data":"f814e22f276b7677be255aa7ed103a6ad14cb1f89dacdc4d76cbcf2126e10e09"}
Sep 18 22:17:33.923377 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:33.923347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:33.923541 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:33.923518    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:17:33.923615 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:33.923604    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:35.923584155 +0000 UTC m=+40.284100996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:17:34.876697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:34.876621    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" event={"ID":"0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9","Type":"ContainerStarted","Data":"813524a5db772ea6e1e6f49485c55e79db59a720229f15440773a6486adc8edc"}
Sep 18 22:17:34.904879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:34.904819    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qjr2x" podStartSLOduration=9.885338581 podStartE2EDuration="37.904799119s" totalImagesPullingTime="28.019460538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:34.901557479 +0000 UTC m=+39.262074338" watchObservedRunningTime="2026-09-18 22:17:34.904799119 +0000 UTC m=+39.265315981"
Sep 18 22:17:35.882346 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:35.882155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hg5t" event={"ID":"309bb642-370f-4c18-896e-844cb37b7672","Type":"ContainerStarted","Data":"1f8b8046d30d0fbd084e324ccdf317f2abcc8bb57680dda20ad791149fceb443"}
Sep 18 22:17:35.899808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:35.899767    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6hg5t" podStartSLOduration=1.431484273 podStartE2EDuration="3.8997522s" totalImagesPullingTime="2.468267927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:35.898800448 +0000 UTC m=+40.259317318" watchObservedRunningTime="2026-09-18 22:17:35.8997522 +0000 UTC m=+40.260269056"
Sep 18 22:17:35.938924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:35.938897    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:35.939058 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:35.939014    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:17:35.939100 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:35.939067    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:39.939050625 +0000 UTC m=+44.299567462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:17:39.966822 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:39.966775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:39.967244 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:39.966923    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:17:39.967244 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:39.966980    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:17:47.966965535 +0000 UTC m=+52.327482370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:17:48.021118 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:48.021075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:17:48.021666 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:48.021228    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:17:48.021666 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:48.021292    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:04.02127441 +0000 UTC m=+68.381791248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:17:54.814644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.814607    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p75rv"]
Sep 18 22:17:54.857282 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.857246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p75rv"]
Sep 18 22:17:54.857442 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.857349    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.860191 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.860166    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 22:17:54.860329 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.860166    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 18 22:17:54.860329 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.860244    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5v8j7\""
Sep 18 22:17:54.861130 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.861114    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 18 22:17:54.861274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.861258    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 22:17:54.865672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.865652    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 18 22:17:54.920872 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.920845    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"]
Sep 18 22:17:54.942973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.942951    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7454c7cb7-zqs7p"]
Sep 18 22:17:54.943087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.943064    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"
Sep 18 22:17:54.945720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.945701    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-m8757\""
Sep 18 22:17:54.945836 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.945771    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:17:54.945836 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.945776    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 18 22:17:54.967031 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.967004    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"]
Sep 18 22:17:54.967031 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.967035    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7454c7cb7-zqs7p"]
Sep 18 22:17:54.967191 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.967109    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:54.969710 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.969691    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 22:17:54.969808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.969692    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 22:17:54.970030 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.970014    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 22:17:54.970103 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.970083    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x79ck\""
Sep 18 22:17:54.970885 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.970867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.970973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.970898    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-serving-cert\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.970973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.970932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zssf4\" (UniqueName: \"kubernetes.io/projected/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-kube-api-access-zssf4\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.971066 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.971013    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.971105 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.971088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-tmp\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.971150 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.971131    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-snapshots\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:54.979842 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:54.979825    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 22:17:55.027157 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.027124    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-p2rff"]
Sep 18 22:17:55.041008 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.040983    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"]
Sep 18 22:17:55.041134 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.041116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.044342 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.044322    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 22:17:55.044465 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.044368    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:17:55.044465 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.044458    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 22:17:55.044836 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.044817    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gkpcg\""
Sep 18 22:17:55.044961 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.044918    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 22:17:55.051289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.051263    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 22:17:55.055027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.055010    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-p2rff"]
Sep 18 22:17:55.055112 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.055039    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"]
Sep 18 22:17:55.055150 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.055120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.057324 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.057307    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 18 22:17:55.057324 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.057314    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:17:55.057481 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.057377    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 18 22:17:55.057638 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.057625    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t5blr\""
Sep 18 22:17:55.071495 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zssf4\" (UniqueName: \"kubernetes.io/projected/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-kube-api-access-zssf4\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.071495 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-certificates\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.071649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-installation-pull-secrets\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.071649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.071649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1938e51f-d5c0-4372-8b9e-4519da5b735a-ca-trust-extracted\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.071649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.071649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-tmp\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.071874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-snapshots\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.071874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5c6z\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-kube-api-access-c5c6z\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.071874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-image-registry-private-configuration\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.071874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-trusted-ca\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.072077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071908    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdl4v\" (UniqueName: \"kubernetes.io/projected/2dd3cdee-7c07-4315-897a-0755cbe63bf4-kube-api-access-pdl4v\") pod \"volume-data-source-validator-5c4db8bdf9-qwnls\" (UID: \"2dd3cdee-7c07-4315-897a-0755cbe63bf4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"
Sep 18 22:17:55.072077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-bound-sa-token\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.072077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071948    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-tmp\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.072077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.071983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.072077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.072012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-serving-cert\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.072315 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.072259    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.072315 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.072308    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-snapshots\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.073069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.073049    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.075083 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.075064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-serving-cert\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.082543 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.082521    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zssf4\" (UniqueName: \"kubernetes.io/projected/484adadf-5f15-4fe6-b88f-0d6e76b0ae3f-kube-api-access-zssf4\") pod \"insights-operator-6f9c644dd9-p75rv\" (UID: \"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f\") " pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.166484 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.166446    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv"
Sep 18 22:17:55.172712 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.172683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e07e370c-b4b7-4974-9297-a24f5647ec3e-serving-cert\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.172823 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.172724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.172823 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.172743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn92r\" (UniqueName: \"kubernetes.io/projected/c25d407d-10a0-40df-9dca-96338052116c-kube-api-access-xn92r\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.172823 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.172809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5c6z\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-kube-api-access-c5c6z\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.172983 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.172962    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 22:17:55.173017 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.172984    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7454c7cb7-zqs7p: secret "image-registry-tls" not found
Sep 18 22:17:55.173046 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.173041    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls podName:1938e51f-d5c0-4372-8b9e-4519da5b735a nodeName:}" failed. No retries permitted until 2026-09-18 22:17:55.673025911 +0000 UTC m=+60.033542749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls") pod "image-registry-7454c7cb7-zqs7p" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a") : secret "image-registry-tls" not found
Sep 18 22:17:55.173089 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.172981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-image-registry-private-configuration\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173089 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173077    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-trusted-ca\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173107    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdl4v\" (UniqueName: \"kubernetes.io/projected/2dd3cdee-7c07-4315-897a-0755cbe63bf4-kube-api-access-pdl4v\") pod \"volume-data-source-validator-5c4db8bdf9-qwnls\" (UID: \"2dd3cdee-7c07-4315-897a-0755cbe63bf4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"
Sep 18 22:17:55.173151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-bound-sa-token\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173223 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.173223 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dvkf\" (UniqueName: \"kubernetes.io/projected/e07e370c-b4b7-4974-9297-a24f5647ec3e-kube-api-access-6dvkf\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.173319 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173235    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-certificates\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173319 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173265    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e07e370c-b4b7-4974-9297-a24f5647ec3e-config\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.173441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-installation-pull-secrets\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1938e51f-d5c0-4372-8b9e-4519da5b735a-ca-trust-extracted\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173541 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e07e370c-b4b7-4974-9297-a24f5647ec3e-trusted-ca\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.173834 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173811    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1938e51f-d5c0-4372-8b9e-4519da5b735a-ca-trust-extracted\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.173907 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.173863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-certificates\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.174371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.174351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-trusted-ca\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.175410 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.175368    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-image-registry-private-configuration\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.175524 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.175508    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-installation-pull-secrets\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.183429 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.183382    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-bound-sa-token\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.183512 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.183451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdl4v\" (UniqueName: \"kubernetes.io/projected/2dd3cdee-7c07-4315-897a-0755cbe63bf4-kube-api-access-pdl4v\") pod \"volume-data-source-validator-5c4db8bdf9-qwnls\" (UID: \"2dd3cdee-7c07-4315-897a-0755cbe63bf4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"
Sep 18 22:17:55.184250 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.184234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5c6z\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-kube-api-access-c5c6z\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.252703 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.252676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"
Sep 18 22:17:55.274726 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.274697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.274843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.274737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dvkf\" (UniqueName: \"kubernetes.io/projected/e07e370c-b4b7-4974-9297-a24f5647ec3e-kube-api-access-6dvkf\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.274843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.274762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e07e370c-b4b7-4974-9297-a24f5647ec3e-config\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.274843 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.274803    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 22:17:55.274965 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.274867    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls podName:c25d407d-10a0-40df-9dca-96338052116c nodeName:}" failed. No retries permitted until 2026-09-18 22:17:55.774851897 +0000 UTC m=+60.135368738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gt9j8" (UID: "c25d407d-10a0-40df-9dca-96338052116c") : secret "samples-operator-tls" not found
Sep 18 22:17:55.274965 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.274809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e07e370c-b4b7-4974-9297-a24f5647ec3e-trusted-ca\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.275046 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.274964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e07e370c-b4b7-4974-9297-a24f5647ec3e-serving-cert\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.275046 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.275012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn92r\" (UniqueName: \"kubernetes.io/projected/c25d407d-10a0-40df-9dca-96338052116c-kube-api-access-xn92r\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.275641 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.275619    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e07e370c-b4b7-4974-9297-a24f5647ec3e-config\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.276002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.275985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e07e370c-b4b7-4974-9297-a24f5647ec3e-trusted-ca\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.277210 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.277193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e07e370c-b4b7-4974-9297-a24f5647ec3e-serving-cert\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.285415 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.285379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dvkf\" (UniqueName: \"kubernetes.io/projected/e07e370c-b4b7-4974-9297-a24f5647ec3e-kube-api-access-6dvkf\") pod \"console-operator-fd4bb46c-p2rff\" (UID: \"e07e370c-b4b7-4974-9297-a24f5647ec3e\") " pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.285623 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.285599    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn92r\" (UniqueName: \"kubernetes.io/projected/c25d407d-10a0-40df-9dca-96338052116c-kube-api-access-xn92r\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.351915 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.351884    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:17:55.454661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.454633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-p75rv"]
Sep 18 22:17:55.457644 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:55.457605    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484adadf_5f15_4fe6_b88f_0d6e76b0ae3f.slice/crio-e1fff7ede156fc9393564b02b29081647876ea78cacc6c5c186e4082a1386c57 WatchSource:0}: Error finding container e1fff7ede156fc9393564b02b29081647876ea78cacc6c5c186e4082a1386c57: Status 404 returned error can't find the container with id e1fff7ede156fc9393564b02b29081647876ea78cacc6c5c186e4082a1386c57
Sep 18 22:17:55.548125 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.548093    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls"]
Sep 18 22:17:55.551165 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:55.551135    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd3cdee_7c07_4315_897a_0755cbe63bf4.slice/crio-a118d4736646e3f32495a8bba1e6afaa5bcbb843dc6dea5eb5c5afe437f37fff WatchSource:0}: Error finding container a118d4736646e3f32495a8bba1e6afaa5bcbb843dc6dea5eb5c5afe437f37fff: Status 404 returned error can't find the container with id a118d4736646e3f32495a8bba1e6afaa5bcbb843dc6dea5eb5c5afe437f37fff
Sep 18 22:17:55.653103 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.653077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-p2rff"]
Sep 18 22:17:55.656064 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:17:55.656040    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07e370c_b4b7_4974_9297_a24f5647ec3e.slice/crio-de1db3fdfcef8d74d22b23f56a0a500e6eb761ba33afc49f6bbb4bd94c1d483c WatchSource:0}: Error finding container de1db3fdfcef8d74d22b23f56a0a500e6eb761ba33afc49f6bbb4bd94c1d483c: Status 404 returned error can't find the container with id de1db3fdfcef8d74d22b23f56a0a500e6eb761ba33afc49f6bbb4bd94c1d483c
Sep 18 22:17:55.679007 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.678981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:55.679162 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.679141    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 22:17:55.679162 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.679161    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7454c7cb7-zqs7p: secret "image-registry-tls" not found
Sep 18 22:17:55.679318 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.679213    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls podName:1938e51f-d5c0-4372-8b9e-4519da5b735a nodeName:}" failed. No retries permitted until 2026-09-18 22:17:56.679199367 +0000 UTC m=+61.039716202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls") pod "image-registry-7454c7cb7-zqs7p" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a") : secret "image-registry-tls" not found
Sep 18 22:17:55.780196 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.780163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:55.780341 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.780280    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 22:17:55.780341 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:55.780333    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls podName:c25d407d-10a0-40df-9dca-96338052116c nodeName:}" failed. No retries permitted until 2026-09-18 22:17:56.780320009 +0000 UTC m=+61.140836857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gt9j8" (UID: "c25d407d-10a0-40df-9dca-96338052116c") : secret "samples-operator-tls" not found
Sep 18 22:17:55.933077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.932995    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" event={"ID":"e07e370c-b4b7-4974-9297-a24f5647ec3e","Type":"ContainerStarted","Data":"de1db3fdfcef8d74d22b23f56a0a500e6eb761ba33afc49f6bbb4bd94c1d483c"}
Sep 18 22:17:55.934174 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.934145    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv" event={"ID":"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f","Type":"ContainerStarted","Data":"e1fff7ede156fc9393564b02b29081647876ea78cacc6c5c186e4082a1386c57"}
Sep 18 22:17:55.935237 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:55.935212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls" event={"ID":"2dd3cdee-7c07-4315-897a-0755cbe63bf4","Type":"ContainerStarted","Data":"a118d4736646e3f32495a8bba1e6afaa5bcbb843dc6dea5eb5c5afe437f37fff"}
Sep 18 22:17:56.689838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:56.689758    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:56.690014 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:56.689951    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 22:17:56.690014 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:56.689976    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7454c7cb7-zqs7p: secret "image-registry-tls" not found
Sep 18 22:17:56.690122 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:56.690045    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls podName:1938e51f-d5c0-4372-8b9e-4519da5b735a nodeName:}" failed. No retries permitted until 2026-09-18 22:17:58.69002264 +0000 UTC m=+63.050539476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls") pod "image-registry-7454c7cb7-zqs7p" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a") : secret "image-registry-tls" not found
Sep 18 22:17:56.790864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:56.790824    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:56.791051 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:56.790997    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 22:17:56.791116 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:56.791075    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls podName:c25d407d-10a0-40df-9dca-96338052116c nodeName:}" failed. No retries permitted until 2026-09-18 22:17:58.79105584 +0000 UTC m=+63.151572689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gt9j8" (UID: "c25d407d-10a0-40df-9dca-96338052116c") : secret "samples-operator-tls" not found
Sep 18 22:17:56.862625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:56.862593    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-km2gr"
Sep 18 22:17:58.705642 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.705612    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:17:58.705997 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:58.705791    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 22:17:58.705997 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:58.705812    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7454c7cb7-zqs7p: secret "image-registry-tls" not found
Sep 18 22:17:58.705997 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:58.705868    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls podName:1938e51f-d5c0-4372-8b9e-4519da5b735a nodeName:}" failed. No retries permitted until 2026-09-18 22:18:02.705848158 +0000 UTC m=+67.066364996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls") pod "image-registry-7454c7cb7-zqs7p" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a") : secret "image-registry-tls" not found
Sep 18 22:17:58.806233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.806200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:17:58.806354 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:58.806334    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 22:17:58.806417 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:58.806409    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls podName:c25d407d-10a0-40df-9dca-96338052116c nodeName:}" failed. No retries permitted until 2026-09-18 22:18:02.80638254 +0000 UTC m=+67.166899380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gt9j8" (UID: "c25d407d-10a0-40df-9dca-96338052116c") : secret "samples-operator-tls" not found
Sep 18 22:17:58.946153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.946121    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/0.log"
Sep 18 22:17:58.946330 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.946163    2583 generic.go:356] "Generic (PLEG): container finished" podID="e07e370c-b4b7-4974-9297-a24f5647ec3e" containerID="a8234b55349b92e167be26190d7907511c6b597611bbd1f351ae6dc6fc2ec5da" exitCode=255
Sep 18 22:17:58.946330 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.946241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" event={"ID":"e07e370c-b4b7-4974-9297-a24f5647ec3e","Type":"ContainerDied","Data":"a8234b55349b92e167be26190d7907511c6b597611bbd1f351ae6dc6fc2ec5da"}
Sep 18 22:17:58.946574 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.946556    2583 scope.go:117] "RemoveContainer" containerID="a8234b55349b92e167be26190d7907511c6b597611bbd1f351ae6dc6fc2ec5da"
Sep 18 22:17:58.947971 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.947946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv" event={"ID":"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f","Type":"ContainerStarted","Data":"ae25b43f263e9b7ba0fd035daad8a04f616fa09f74e81b8204f210d2c53fefa5"}
Sep 18 22:17:58.949669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.949644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls" event={"ID":"2dd3cdee-7c07-4315-897a-0755cbe63bf4","Type":"ContainerStarted","Data":"e798cfe1c0d98d1a658b665506a15022640f8c7ae37fe644075733cf29bdc890"}
Sep 18 22:17:58.980622 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:58.980568    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-qwnls" podStartSLOduration=1.9527843950000001 podStartE2EDuration="4.980552116s" totalImagesPullingTime="3.027767721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:58.980443622 +0000 UTC m=+63.340960478" watchObservedRunningTime="2026-09-18 22:17:58.980552116 +0000 UTC m=+63.341068975"
Sep 18 22:17:59.000712 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.000672    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv" podStartSLOduration=1.861982814 podStartE2EDuration="5.000658519s" totalImagesPullingTime="3.138675705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:17:59.000203505 +0000 UTC m=+63.360720394" watchObservedRunningTime="2026-09-18 22:17:59.000658519 +0000 UTC m=+63.361175376"
Sep 18 22:17:59.954357 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.954328    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:17:59.954793 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.954758    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/0.log"
Sep 18 22:17:59.954844 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.954790    2583 generic.go:356] "Generic (PLEG): container finished" podID="e07e370c-b4b7-4974-9297-a24f5647ec3e" containerID="030b864895767c625692ab387710e4acf8355c5fe97031ea1ba4dfff4e4a71d1" exitCode=255
Sep 18 22:17:59.954904 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.954878    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" event={"ID":"e07e370c-b4b7-4974-9297-a24f5647ec3e","Type":"ContainerDied","Data":"030b864895767c625692ab387710e4acf8355c5fe97031ea1ba4dfff4e4a71d1"}
Sep 18 22:17:59.954957 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.954936    2583 scope.go:117] "RemoveContainer" containerID="a8234b55349b92e167be26190d7907511c6b597611bbd1f351ae6dc6fc2ec5da"
Sep 18 22:17:59.955171 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:17:59.955139    2583 scope.go:117] "RemoveContainer" containerID="030b864895767c625692ab387710e4acf8355c5fe97031ea1ba4dfff4e4a71d1"
Sep 18 22:17:59.955359 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:17:59.955340    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-p2rff_openshift-console-operator(e07e370c-b4b7-4974-9297-a24f5647ec3e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" podUID="e07e370c-b4b7-4974-9297-a24f5647ec3e"
Sep 18 22:18:00.195201 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.195169    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rbfcg"]
Sep 18 22:18:00.226904 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.226823    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rbfcg"]
Sep 18 22:18:00.227065 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.226976    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.229296 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.229268    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 22:18:00.229488 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.229472    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 22:18:00.229545 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.229538    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7xc5k\""
Sep 18 22:18:00.318034 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.317999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvkzl\" (UniqueName: \"kubernetes.io/projected/53d716ef-46dc-4722-8340-869103572c25-kube-api-access-wvkzl\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.318184 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.318042    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.318184 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.318085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.318184 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.318104    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53d716ef-46dc-4722-8340-869103572c25-crio-socket\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.318283 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.318186    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53d716ef-46dc-4722-8340-869103572c25-data-volume\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419507 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419516    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53d716ef-46dc-4722-8340-869103572c25-crio-socket\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419602    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53d716ef-46dc-4722-8340-869103572c25-crio-socket\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419656 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.419615    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 22:18:00.419656 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.419626    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:00.419821 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53d716ef-46dc-4722-8340-869103572c25-data-volume\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419821 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.419682    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:00.919662894 +0000 UTC m=+65.280179739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:00.419821 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvkzl\" (UniqueName: \"kubernetes.io/projected/53d716ef-46dc-4722-8340-869103572c25-kube-api-access-wvkzl\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.419821 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.419777    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:00.919755275 +0000 UTC m=+65.280272134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : configmap "kube-rbac-proxy" not found
Sep 18 22:18:00.419961 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.419929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53d716ef-46dc-4722-8340-869103572c25-data-volume\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.431083 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.431056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvkzl\" (UniqueName: \"kubernetes.io/projected/53d716ef-46dc-4722-8340-869103572c25-kube-api-access-wvkzl\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.924559 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.924524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.924731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.924570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:00.924731 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.924677    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:00.924731 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.924706    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 22:18:00.924830 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.924740    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:01.924723405 +0000 UTC m=+66.285240256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:00.924830 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.924764    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:01.924752581 +0000 UTC m=+66.285269433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : configmap "kube-rbac-proxy" not found
Sep 18 22:18:00.961956 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.961930    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:18:00.962321 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:00.962226    2583 scope.go:117] "RemoveContainer" containerID="030b864895767c625692ab387710e4acf8355c5fe97031ea1ba4dfff4e4a71d1"
Sep 18 22:18:00.962401 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:00.962369    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-p2rff_openshift-console-operator(e07e370c-b4b7-4974-9297-a24f5647ec3e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" podUID="e07e370c-b4b7-4974-9297-a24f5647ec3e"
Sep 18 22:18:01.933610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:01.933580    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:01.933788 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:01.933626    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:01.933788 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:01.933698    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 22:18:01.933788 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:01.933731    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:01.933788 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:01.933741    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:03.9337285 +0000 UTC m=+68.294245336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : configmap "kube-rbac-proxy" not found
Sep 18 22:18:01.933952 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:01.933841    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:03.933829379 +0000 UTC m=+68.294346215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:02.111297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:02.111272    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r28kg_b39dd412-80b2-4d98-b06a-8c0f1b86a59b/dns-node-resolver/0.log"
Sep 18 22:18:02.739080 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:02.739039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:02.739263 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:02.739190    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 22:18:02.739263 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:02.739209    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7454c7cb7-zqs7p: secret "image-registry-tls" not found
Sep 18 22:18:02.739263 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:02.739262    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls podName:1938e51f-d5c0-4372-8b9e-4519da5b735a nodeName:}" failed. No retries permitted until 2026-09-18 22:18:10.739248309 +0000 UTC m=+75.099765162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls") pod "image-registry-7454c7cb7-zqs7p" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a") : secret "image-registry-tls" not found
Sep 18 22:18:02.840215 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:02.840180    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:18:02.840359 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:02.840321    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 22:18:02.840418 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:02.840382    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls podName:c25d407d-10a0-40df-9dca-96338052116c nodeName:}" failed. No retries permitted until 2026-09-18 22:18:10.840367021 +0000 UTC m=+75.200883855 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-gt9j8" (UID: "c25d407d-10a0-40df-9dca-96338052116c") : secret "samples-operator-tls" not found
Sep 18 22:18:02.911636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:02.911607    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gdk85_8672980a-1096-46f2-a1ba-d5d1c768e247/node-ca/0.log"
Sep 18 22:18:03.343958 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.343923    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:18:03.346825 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.346804    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 22:18:03.356693 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.356673    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 22:18:03.366860 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.366840    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxdgf\" (UniqueName: \"kubernetes.io/projected/6c7417ee-d304-4d98-9ade-8db75a074ea9-kube-api-access-vxdgf\") pod \"network-check-target-lwvjn\" (UID: \"6c7417ee-d304-4d98-9ade-8db75a074ea9\") " pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:18:03.444785 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.444752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:18:03.444943 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.444830    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:18:03.447287 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.447266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 22:18:03.447486 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.447468    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 22:18:03.455956 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:03.455939    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 22:18:03.456009 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:03.455999    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs podName:2f3ceffc-4830-463b-ab1e-8d1880fade2e nodeName:}" failed. No retries permitted until 2026-09-18 22:19:07.455979723 +0000 UTC m=+131.816496570 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs") pod "network-metrics-daemon-xzzt2" (UID: "2f3ceffc-4830-463b-ab1e-8d1880fade2e") : secret "metrics-daemon-secret" not found
Sep 18 22:18:03.458041 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.458024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a26f907f-1903-4bf9-b515-62df85b2b6f7-original-pull-secret\") pod \"global-pull-secret-syncer-4t6gh\" (UID: \"a26f907f-1903-4bf9-b515-62df85b2b6f7\") " pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:18:03.586886 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.586852    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4t6gh"
Sep 18 22:18:03.596460 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.596426    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-95pvs\""
Sep 18 22:18:03.602534 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.602515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:18:03.718342 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.718317    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hg5t_309bb642-370f-4c18-896e-844cb37b7672/serve-healthcheck-canary/0.log"
Sep 18 22:18:03.860805 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.860729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4t6gh"]
Sep 18 22:18:03.863924 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:03.863892    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26f907f_1903_4bf9_b515_62df85b2b6f7.slice/crio-570955ddc7c79c127136bec32d623fe9c0bf6581527cca94729f93ba126e99fb WatchSource:0}: Error finding container 570955ddc7c79c127136bec32d623fe9c0bf6581527cca94729f93ba126e99fb: Status 404 returned error can't find the container with id 570955ddc7c79c127136bec32d623fe9c0bf6581527cca94729f93ba126e99fb
Sep 18 22:18:03.879482 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.879459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lwvjn"]
Sep 18 22:18:03.882420 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:03.882376    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7417ee_d304_4d98_9ade_8db75a074ea9.slice/crio-893c208df98d4b94698841f4eb14ed2915e89e6e4f5ca7ce0d554b63aa5eb926 WatchSource:0}: Error finding container 893c208df98d4b94698841f4eb14ed2915e89e6e4f5ca7ce0d554b63aa5eb926: Status 404 returned error can't find the container with id 893c208df98d4b94698841f4eb14ed2915e89e6e4f5ca7ce0d554b63aa5eb926
Sep 18 22:18:03.949669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.949632    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:03.949803 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.949747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:03.949803 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:03.949750    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 22:18:03.949891 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:03.949809    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:07.949793609 +0000 UTC m=+72.310310443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : configmap "kube-rbac-proxy" not found
Sep 18 22:18:03.949891 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:03.949845    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:03.949891 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:03.949883    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:07.949872862 +0000 UTC m=+72.310389716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:03.970588 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.970561    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lwvjn" event={"ID":"6c7417ee-d304-4d98-9ade-8db75a074ea9","Type":"ContainerStarted","Data":"893c208df98d4b94698841f4eb14ed2915e89e6e4f5ca7ce0d554b63aa5eb926"}
Sep 18 22:18:03.971747 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:03.971725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4t6gh" event={"ID":"a26f907f-1903-4bf9-b515-62df85b2b6f7","Type":"ContainerStarted","Data":"570955ddc7c79c127136bec32d623fe9c0bf6581527cca94729f93ba126e99fb"}
Sep 18 22:18:04.050827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:04.050799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:18:04.050962 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:04.050944    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 22:18:04.051014 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:04.051005    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls podName:c843f1a2-c11e-4e96-aa81-998169823479 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:36.050990218 +0000 UTC m=+100.411507057 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls") pod "dns-default-sqsj6" (UID: "c843f1a2-c11e-4e96-aa81-998169823479") : secret "dns-default-metrics-tls" not found
Sep 18 22:18:05.353093 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:05.353050    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:18:05.353685 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:05.353120    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:18:05.353685 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:05.353618    2583 scope.go:117] "RemoveContainer" containerID="030b864895767c625692ab387710e4acf8355c5fe97031ea1ba4dfff4e4a71d1"
Sep 18 22:18:05.353896 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:05.353845    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-p2rff_openshift-console-operator(e07e370c-b4b7-4974-9297-a24f5647ec3e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" podUID="e07e370c-b4b7-4974-9297-a24f5647ec3e"
Sep 18 22:18:07.987001 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:07.986965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:07.987468 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:07.987026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:07.987468 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:07.987102    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 22:18:07.987468 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:07.987135    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:07.987468 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:07.987166    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:15.987147635 +0000 UTC m=+80.347664471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : configmap "kube-rbac-proxy" not found
Sep 18 22:18:07.987468 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:07.987200    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls podName:53d716ef-46dc-4722-8340-869103572c25 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:15.987181813 +0000 UTC m=+80.347698665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rbfcg" (UID: "53d716ef-46dc-4722-8340-869103572c25") : secret "insights-runtime-extractor-tls" not found
Sep 18 22:18:08.988368 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:08.988335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lwvjn" event={"ID":"6c7417ee-d304-4d98-9ade-8db75a074ea9","Type":"ContainerStarted","Data":"5cc154634424e24f8ad4fd379694702c161ede74b8125bc411f4f8b7d7d49b53"}
Sep 18 22:18:08.988781 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:08.988451    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:18:08.989890 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:08.989870    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4t6gh" event={"ID":"a26f907f-1903-4bf9-b515-62df85b2b6f7","Type":"ContainerStarted","Data":"9eef6cf5d84ad113eecdaa5c40311b7c7f0a79ad85e26a582a96a68a70702bd8"}
Sep 18 22:18:09.013426 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:09.013362    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lwvjn" podStartSLOduration=67.763458417 podStartE2EDuration="1m12.013350742s" totalImagesPullingTime="4.249892325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:09.012913724 +0000 UTC m=+73.373430581" watchObservedRunningTime="2026-09-18 22:18:09.013350742 +0000 UTC m=+73.373867633"
Sep 18 22:18:09.047980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:09.047937    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4t6gh" podStartSLOduration=67.775271011 podStartE2EDuration="1m12.047915933s" totalImagesPullingTime="4.272644922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:09.046541166 +0000 UTC m=+73.407058023" watchObservedRunningTime="2026-09-18 22:18:09.047915933 +0000 UTC m=+73.408432787"
Sep 18 22:18:10.809435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.809291    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:10.811610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.811590    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"image-registry-7454c7cb7-zqs7p\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") " pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:10.879062 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.879028    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x79ck\""
Sep 18 22:18:10.887216 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.887195    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:10.910697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.910652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:18:10.912962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.912936    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c25d407d-10a0-40df-9dca-96338052116c-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-gt9j8\" (UID: \"c25d407d-10a0-40df-9dca-96338052116c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:18:10.965959 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.965932    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t5blr\""
Sep 18 22:18:10.973595 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:10.973573    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"
Sep 18 22:18:11.163225 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:11.163120    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7454c7cb7-zqs7p"]
Sep 18 22:18:11.165600 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:11.165569    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1938e51f_d5c0_4372_8b9e_4519da5b735a.slice/crio-969cf6187eeca3b8b6a639165100d761bd2d043ce001cd22be67cd65e5bcf53e WatchSource:0}: Error finding container 969cf6187eeca3b8b6a639165100d761bd2d043ce001cd22be67cd65e5bcf53e: Status 404 returned error can't find the container with id 969cf6187eeca3b8b6a639165100d761bd2d043ce001cd22be67cd65e5bcf53e
Sep 18 22:18:11.241273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:11.241244    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8"]
Sep 18 22:18:12.002348 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:12.002303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8" event={"ID":"c25d407d-10a0-40df-9dca-96338052116c","Type":"ContainerStarted","Data":"5e5aa4a737b0cff3814e4a491ff46b39141d73944dad4cb100ccafa8d7b56b81"}
Sep 18 22:18:12.004299 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:12.004272    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" event={"ID":"1938e51f-d5c0-4372-8b9e-4519da5b735a","Type":"ContainerStarted","Data":"06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde"}
Sep 18 22:18:12.004462 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:12.004318    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" event={"ID":"1938e51f-d5c0-4372-8b9e-4519da5b735a","Type":"ContainerStarted","Data":"969cf6187eeca3b8b6a639165100d761bd2d043ce001cd22be67cd65e5bcf53e"}
Sep 18 22:18:12.004462 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:12.004437    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:12.025190 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:12.025138    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" podStartSLOduration=18.025122528 podStartE2EDuration="18.025122528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:12.024130662 +0000 UTC m=+76.384647519" watchObservedRunningTime="2026-09-18 22:18:12.025122528 +0000 UTC m=+76.385639385"
Sep 18 22:18:13.009606 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:13.009570    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8" event={"ID":"c25d407d-10a0-40df-9dca-96338052116c","Type":"ContainerStarted","Data":"2fa3bc7889338933f50093f20cbc8e1ddd31fc1d3b1b981cce2eab8ae4692377"}
Sep 18 22:18:14.014354 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:14.014320    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8" event={"ID":"c25d407d-10a0-40df-9dca-96338052116c","Type":"ContainerStarted","Data":"885c3bce09114e5662f9000730621b25b3837a6372cabad41620a35220a68257"}
Sep 18 22:18:14.030504 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:14.030456    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-gt9j8" podStartSLOduration=18.422051612 podStartE2EDuration="20.030436876s" totalImagesPullingTime="1.608385264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:14.029838436 +0000 UTC m=+78.390355302" watchObservedRunningTime="2026-09-18 22:18:14.030436876 +0000 UTC m=+78.390953733"
Sep 18 22:18:16.052469 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:16.052434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:16.052873 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:16.052478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:16.052935 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:16.052918    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53d716ef-46dc-4722-8340-869103572c25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:16.054648 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:16.054630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53d716ef-46dc-4722-8340-869103572c25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbfcg\" (UID: \"53d716ef-46dc-4722-8340-869103572c25\") " pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:16.136417 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:16.136366    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rbfcg"
Sep 18 22:18:16.417778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:16.417750    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rbfcg"]
Sep 18 22:18:16.421073 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:16.421047    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d716ef_46dc_4722_8340_869103572c25.slice/crio-15b94d300e62271bce9766747515b68de1c8f8de742a656c0dff7316cbf812bd WatchSource:0}: Error finding container 15b94d300e62271bce9766747515b68de1c8f8de742a656c0dff7316cbf812bd: Status 404 returned error can't find the container with id 15b94d300e62271bce9766747515b68de1c8f8de742a656c0dff7316cbf812bd
Sep 18 22:18:17.023745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:17.023704    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbfcg" event={"ID":"53d716ef-46dc-4722-8340-869103572c25","Type":"ContainerStarted","Data":"a81dfb8b929abf9ace84e4b60a38f9e8b1dfe6092db75d4744c1845ed1d7cfca"}
Sep 18 22:18:17.023893 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:17.023759    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbfcg" event={"ID":"53d716ef-46dc-4722-8340-869103572c25","Type":"ContainerStarted","Data":"15b94d300e62271bce9766747515b68de1c8f8de742a656c0dff7316cbf812bd"}
Sep 18 22:18:17.678022 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:17.677995    2583 scope.go:117] "RemoveContainer" containerID="030b864895767c625692ab387710e4acf8355c5fe97031ea1ba4dfff4e4a71d1"
Sep 18 22:18:18.029534 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:18.029454    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:18:18.029699 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:18.029570    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" event={"ID":"e07e370c-b4b7-4974-9297-a24f5647ec3e","Type":"ContainerStarted","Data":"4bd7525774ed975c800d98c97d58bf33a4cb11497ba2c34182f4c3d16a10e1c2"}
Sep 18 22:18:18.029960 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:18.029940    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:18:18.031891 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:18.031866    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbfcg" event={"ID":"53d716ef-46dc-4722-8340-869103572c25","Type":"ContainerStarted","Data":"d304e48a6c49f0e0a9a172678e41d0ad35bba96063a6f7a8f2e8a23a3c25fbe7"}
Sep 18 22:18:18.047733 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:18.047695    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff" podStartSLOduration=21.102722474 podStartE2EDuration="24.047680078s" totalImagesPullingTime="2.944957604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:18.047270077 +0000 UTC m=+82.407786935" watchObservedRunningTime="2026-09-18 22:18:18.047680078 +0000 UTC m=+82.408196937"
Sep 18 22:18:18.175067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:18.175038    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-p2rff"
Sep 18 22:18:19.036966 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:19.036928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbfcg" event={"ID":"53d716ef-46dc-4722-8340-869103572c25","Type":"ContainerStarted","Data":"a871497ec770b319b41db65bae5d86fa524126cabc302186acde8ed0bcc05877"}
Sep 18 22:18:23.055227 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.055167    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rbfcg" podStartSLOduration=21.042098648 podStartE2EDuration="23.055152355s" totalImagesPullingTime="2.013053707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:18:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:19.057294101 +0000 UTC m=+83.417810983" watchObservedRunningTime="2026-09-18 22:18:23.055152355 +0000 UTC m=+87.415669211"
Sep 18 22:18:23.056518 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.056493    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"]
Sep 18 22:18:23.059990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.059973    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.063664 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.063646    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 22:18:23.067047 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.067032    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 18 22:18:23.068076 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.068024    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 22:18:23.068076 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.068063    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 18 22:18:23.068076 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.068076    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 22:18:23.068292 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.068138    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 18 22:18:23.068292 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.068259    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 18 22:18:23.069421 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.069402    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7454c7cb7-zqs7p"]
Sep 18 22:18:23.081695 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.081672    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"]
Sep 18 22:18:23.206437 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.206401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.206437 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.206439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-ca\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.206649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.206494    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lgrx\" (UniqueName: \"kubernetes.io/projected/350b95eb-cb56-43cd-9c95-3d526d762d9e-kube-api-access-6lgrx\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.206649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.206535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.206649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.206557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/350b95eb-cb56-43cd-9c95-3d526d762d9e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.206649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.206596    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-hub\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.307861 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.307767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.307861 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.307809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-ca\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.308096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.307928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lgrx\" (UniqueName: \"kubernetes.io/projected/350b95eb-cb56-43cd-9c95-3d526d762d9e-kube-api-access-6lgrx\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.308096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.307982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.308096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.308009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/350b95eb-cb56-43cd-9c95-3d526d762d9e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.308096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.308062    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-hub\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.308825 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.308796    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/350b95eb-cb56-43cd-9c95-3d526d762d9e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.310444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.310420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-ca\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.310572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.310500    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-hub\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.310572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.310528    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.310675 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.310659    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/350b95eb-cb56-43cd-9c95-3d526d762d9e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.320043 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.320024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lgrx\" (UniqueName: \"kubernetes.io/projected/350b95eb-cb56-43cd-9c95-3d526d762d9e-kube-api-access-6lgrx\") pod \"cluster-proxy-proxy-agent-664fd98bf-5wg9j\" (UID: \"350b95eb-cb56-43cd-9c95-3d526d762d9e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.387195 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.387166    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"
Sep 18 22:18:23.665564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:23.665532    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j"]
Sep 18 22:18:23.668344 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:23.668319    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod350b95eb_cb56_43cd_9c95_3d526d762d9e.slice/crio-752f19f85a910beb8ebf399ec877527925b202c58e248650aa16dd293cd65c32 WatchSource:0}: Error finding container 752f19f85a910beb8ebf399ec877527925b202c58e248650aa16dd293cd65c32: Status 404 returned error can't find the container with id 752f19f85a910beb8ebf399ec877527925b202c58e248650aa16dd293cd65c32
Sep 18 22:18:24.054459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:24.054369    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j" event={"ID":"350b95eb-cb56-43cd-9c95-3d526d762d9e","Type":"ContainerStarted","Data":"752f19f85a910beb8ebf399ec877527925b202c58e248650aa16dd293cd65c32"}
Sep 18 22:18:28.072563 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:28.072530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j" event={"ID":"350b95eb-cb56-43cd-9c95-3d526d762d9e","Type":"ContainerStarted","Data":"f1c6fbda9bcb7db192c510c9bf1b6c2551ededd8cdb05f3c12e0a50e941b0569"}
Sep 18 22:18:28.073024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:28.072575    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j" event={"ID":"350b95eb-cb56-43cd-9c95-3d526d762d9e","Type":"ContainerStarted","Data":"df130458e7801d036f77e459cbbd8d3716f7c17c02221c2465cf4507236795d4"}
Sep 18 22:18:28.073024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:28.072593    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j" event={"ID":"350b95eb-cb56-43cd-9c95-3d526d762d9e","Type":"ContainerStarted","Data":"c96922852a23c1ae6648b265bc88b6f816ae8bcc50e100dd9220555ac6b2e686"}
Sep 18 22:18:28.096092 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:28.096050    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664fd98bf-5wg9j" podStartSLOduration=1.534644326 podStartE2EDuration="5.096035643s" totalImagesPullingTime="3.561391317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:28.096002687 +0000 UTC m=+92.456519524" watchObservedRunningTime="2026-09-18 22:18:28.096035643 +0000 UTC m=+92.456552494"
Sep 18 22:18:33.074677 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:33.074639    2583 patch_prober.go:28] interesting pod/image-registry-7454c7cb7-zqs7p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 22:18:33.075050 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:33.074692    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 22:18:36.110104 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:36.110059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:18:36.112347 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:36.112321    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c843f1a2-c11e-4e96-aa81-998169823479-metrics-tls\") pod \"dns-default-sqsj6\" (UID: \"c843f1a2-c11e-4e96-aa81-998169823479\") " pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:18:36.131940 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:36.131917    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gtntj\""
Sep 18 22:18:36.140209 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:36.140187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:18:36.410178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:36.410147    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sqsj6"]
Sep 18 22:18:36.412888 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:36.412850    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc843f1a2_c11e_4e96_aa81_998169823479.slice/crio-0e8daae5e4faacf9cebef3056af91a557d44e859d6e16b187d40760d3f962f47 WatchSource:0}: Error finding container 0e8daae5e4faacf9cebef3056af91a557d44e859d6e16b187d40760d3f962f47: Status 404 returned error can't find the container with id 0e8daae5e4faacf9cebef3056af91a557d44e859d6e16b187d40760d3f962f47
Sep 18 22:18:37.104064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:37.104024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sqsj6" event={"ID":"c843f1a2-c11e-4e96-aa81-998169823479","Type":"ContainerStarted","Data":"0e8daae5e4faacf9cebef3056af91a557d44e859d6e16b187d40760d3f962f47"}
Sep 18 22:18:38.109314 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:38.109235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sqsj6" event={"ID":"c843f1a2-c11e-4e96-aa81-998169823479","Type":"ContainerStarted","Data":"c93cc2fe77da5027cd175fdd72bbfe5e79eeaf86e1a8fc464c2be607894d16be"}
Sep 18 22:18:38.109314 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:38.109283    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sqsj6" event={"ID":"c843f1a2-c11e-4e96-aa81-998169823479","Type":"ContainerStarted","Data":"ed53eea3e9adde9be89f6f0e05939a866184ad138e73c5b10f0c8be0ce8d105a"}
Sep 18 22:18:38.109773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:38.109321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:18:38.129855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:38.129807    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sqsj6" podStartSLOduration=64.704734023 podStartE2EDuration="1m6.129792844s" totalImagesPullingTime="1.425058821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:38.128296489 +0000 UTC m=+102.488813347" watchObservedRunningTime="2026-09-18 22:18:38.129792844 +0000 UTC m=+102.490309724"
Sep 18 22:18:39.995589 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:39.995560    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lwvjn"
Sep 18 22:18:43.073726 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:43.073695    2583 patch_prober.go:28] interesting pod/image-registry-7454c7cb7-zqs7p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 22:18:43.074196 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:43.073746    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 22:18:44.817139 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.817103    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dkqq7"]
Sep 18 22:18:44.819815 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.819794    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.822398 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.822357    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-825pr\""
Sep 18 22:18:44.822804 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.822658    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 22:18:44.823314 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.823292    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 22:18:44.823447 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.823331    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 22:18:44.823447 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.823423    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 22:18:44.823556 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.823477    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 22:18:44.823556 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.823525    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 22:18:44.976961 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.976933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-sys\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977114 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.976968    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977114 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.976994    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-textfile\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977114 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.977050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-tls\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977114 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.977090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-wtmp\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977114 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.977110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-accelerators-collector-config\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.977133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-root\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.977168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6344af3b-75e5-4207-9e55-39fcf7477705-metrics-client-ca\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:44.977273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:44.977188    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98v6f\" (UniqueName: \"kubernetes.io/projected/6344af3b-75e5-4207-9e55-39fcf7477705-kube-api-access-98v6f\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078437 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078335    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-sys\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078437 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078374    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-sys\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-textfile\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078558    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-tls\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-wtmp\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078612    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-accelerators-collector-config\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078640    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-root\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6344af3b-75e5-4207-9e55-39fcf7477705-metrics-client-ca\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98v6f\" (UniqueName: \"kubernetes.io/projected/6344af3b-75e5-4207-9e55-39fcf7477705-kube-api-access-98v6f\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:45.078764    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078799    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-root\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078762    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-wtmp\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.078851 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:45.078835    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-tls podName:6344af3b-75e5-4207-9e55-39fcf7477705 nodeName:}" failed. No retries permitted until 2026-09-18 22:18:45.578813113 +0000 UTC m=+109.939329959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-tls") pod "node-exporter-dkqq7" (UID: "6344af3b-75e5-4207-9e55-39fcf7477705") : secret "node-exporter-tls" not found
Sep 18 22:18:45.079132 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.078859    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-textfile\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.079216 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.079196    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-accelerators-collector-config\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.079276 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.079260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6344af3b-75e5-4207-9e55-39fcf7477705-metrics-client-ca\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.080727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.080711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.087634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.087613    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98v6f\" (UniqueName: \"kubernetes.io/projected/6344af3b-75e5-4207-9e55-39fcf7477705-kube-api-access-98v6f\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.581982 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.581948    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-tls\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.584258 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.584223    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6344af3b-75e5-4207-9e55-39fcf7477705-node-exporter-tls\") pod \"node-exporter-dkqq7\" (UID: \"6344af3b-75e5-4207-9e55-39fcf7477705\") " pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.729064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:45.729031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dkqq7"
Sep 18 22:18:45.741442 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:45.741406    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6344af3b_75e5_4207_9e55_39fcf7477705.slice/crio-fb94ce3c14dfc4f7cf7f256e78b45556fb7825eb9ec0dcd569b3c782915deab5 WatchSource:0}: Error finding container fb94ce3c14dfc4f7cf7f256e78b45556fb7825eb9ec0dcd569b3c782915deab5: Status 404 returned error can't find the container with id fb94ce3c14dfc4f7cf7f256e78b45556fb7825eb9ec0dcd569b3c782915deab5
Sep 18 22:18:46.138551 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:46.138511    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkqq7" event={"ID":"6344af3b-75e5-4207-9e55-39fcf7477705","Type":"ContainerStarted","Data":"fb94ce3c14dfc4f7cf7f256e78b45556fb7825eb9ec0dcd569b3c782915deab5"}
Sep 18 22:18:47.143465 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:47.143434    2583 generic.go:356] "Generic (PLEG): container finished" podID="6344af3b-75e5-4207-9e55-39fcf7477705" containerID="9f6dc22faf566c7b5e8bf1f46333ab994694741228001308dc163e87f35ede66" exitCode=0
Sep 18 22:18:47.143806 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:47.143502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkqq7" event={"ID":"6344af3b-75e5-4207-9e55-39fcf7477705","Type":"ContainerDied","Data":"9f6dc22faf566c7b5e8bf1f46333ab994694741228001308dc163e87f35ede66"}
Sep 18 22:18:48.088229 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.088166    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerName="registry" containerID="cri-o://06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde" gracePeriod=30
Sep 18 22:18:48.115071 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.115047    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sqsj6"
Sep 18 22:18:48.150083 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.150044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkqq7" event={"ID":"6344af3b-75e5-4207-9e55-39fcf7477705","Type":"ContainerStarted","Data":"4f776b6dbeac72ec825ea6530596edc5168aa4a60fbb5b7a47789f0e90cbbddc"}
Sep 18 22:18:48.150472 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.150115    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkqq7" event={"ID":"6344af3b-75e5-4207-9e55-39fcf7477705","Type":"ContainerStarted","Data":"61395df6378ab610506cf0826deaf64075c6e37dac7a2868eb653b27e84a4e79"}
Sep 18 22:18:48.170006 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.169960    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dkqq7" podStartSLOduration=3.145043567 podStartE2EDuration="4.169945948s" totalImagesPullingTime="965.390465ms" totalInitContainerRuntime="59.511916ms" isStatefulPod=true podCreationTimestamp="2026-09-18 22:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:48.168706879 +0000 UTC m=+112.529223736" watchObservedRunningTime="2026-09-18 22:18:48.169945948 +0000 UTC m=+112.530462804"
Sep 18 22:18:48.330981 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.330953    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:48.505443 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505404    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-trusted-ca\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505498    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-image-registry-private-configuration\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505537    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5c6z\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-kube-api-access-c5c6z\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505565    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-certificates\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505598    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1938e51f-d5c0-4372-8b9e-4519da5b735a-ca-trust-extracted\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505884 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505625    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-bound-sa-token\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505884 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505680    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-installation-pull-secrets\") pod \"1938e51f-d5c0-4372-8b9e-4519da5b735a\" (UID: \"1938e51f-d5c0-4372-8b9e-4519da5b735a\") "
Sep 18 22:18:48.505988 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.505933    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:18:48.506112 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.506079    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:18:48.508206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.508165    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-kube-api-access-c5c6z" (OuterVolumeSpecName: "kube-api-access-c5c6z") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "kube-api-access-c5c6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:18:48.508206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.508192    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:18:48.508375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.508271    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:18:48.508375 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.508353    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:18:48.508500 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.508450    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:18:48.514677 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.514653    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1938e51f-d5c0-4372-8b9e-4519da5b735a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1938e51f-d5c0-4372-8b9e-4519da5b735a" (UID: "1938e51f-d5c0-4372-8b9e-4519da5b735a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:18:48.606709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606660    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-installation-pull-secrets\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606703    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-trusted-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606714    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606723    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1938e51f-d5c0-4372-8b9e-4519da5b735a-image-registry-private-configuration\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606733    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5c6z\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-kube-api-access-c5c6z\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606743    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1938e51f-d5c0-4372-8b9e-4519da5b735a-registry-certificates\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606751    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1938e51f-d5c0-4372-8b9e-4519da5b735a-ca-trust-extracted\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:48.606969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:48.606759    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1938e51f-d5c0-4372-8b9e-4519da5b735a-bound-sa-token\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:18:49.154986 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.154950    2583 generic.go:356] "Generic (PLEG): container finished" podID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerID="06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde" exitCode=0
Sep 18 22:18:49.155473 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.155002    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p"
Sep 18 22:18:49.155473 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.155038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" event={"ID":"1938e51f-d5c0-4372-8b9e-4519da5b735a","Type":"ContainerDied","Data":"06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde"}
Sep 18 22:18:49.155473 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.155093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7454c7cb7-zqs7p" event={"ID":"1938e51f-d5c0-4372-8b9e-4519da5b735a","Type":"ContainerDied","Data":"969cf6187eeca3b8b6a639165100d761bd2d043ce001cd22be67cd65e5bcf53e"}
Sep 18 22:18:49.155473 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.155114    2583 scope.go:117] "RemoveContainer" containerID="06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde"
Sep 18 22:18:49.163964 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.163948    2583 scope.go:117] "RemoveContainer" containerID="06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde"
Sep 18 22:18:49.164221 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:18:49.164201    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde\": container with ID starting with 06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde not found: ID does not exist" containerID="06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde"
Sep 18 22:18:49.164274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.164246    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde"} err="failed to get container status \"06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde\": rpc error: code = NotFound desc = could not find container \"06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde\": container with ID starting with 06b3f039fa45630e1676725a79b457e383f682f9b03ec96065cce96c6ab28fde not found: ID does not exist"
Sep 18 22:18:49.179236 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.179214    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7454c7cb7-zqs7p"]
Sep 18 22:18:49.182999 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.182978    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7454c7cb7-zqs7p"]
Sep 18 22:18:49.224433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.224408    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c49ff9684-plf67"]
Sep 18 22:18:49.224724 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.224708    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerName="registry"
Sep 18 22:18:49.224799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.224726    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerName="registry"
Sep 18 22:18:49.224799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.224792    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" containerName="registry"
Sep 18 22:18:49.230517 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.230500    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.233121 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.233097    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 22:18:49.233220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.233146    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lcpmb\""
Sep 18 22:18:49.233220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.233184    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 22:18:49.233456 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.233441    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 22:18:49.233456 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.233452    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 22:18:49.233611 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.233595    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d2mfekvbrpirl\""
Sep 18 22:18:49.238185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.238166    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c49ff9684-plf67"]
Sep 18 22:18:49.412731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-metrics-server-audit-profiles\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.412731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-audit-log\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.412731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f58fz\" (UniqueName: \"kubernetes.io/projected/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-kube-api-access-f58fz\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.412939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412772    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-client-ca-bundle\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.412939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412789    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-secret-metrics-server-tls\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.412939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.412939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.412883    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-secret-metrics-server-client-certs\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514221 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-client-ca-bundle\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-secret-metrics-server-tls\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-secret-metrics-server-client-certs\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514336    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-metrics-server-audit-profiles\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-audit-log\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.514685 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f58fz\" (UniqueName: \"kubernetes.io/projected/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-kube-api-access-f58fz\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.515021 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.514935    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-audit-log\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.515135 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.515029    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.515555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.515530    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-metrics-server-audit-profiles\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.516802 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.516781    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-client-ca-bundle\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.517057 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.517039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-secret-metrics-server-client-certs\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.517105 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.517039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-secret-metrics-server-tls\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.522293 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.522263    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f58fz\" (UniqueName: \"kubernetes.io/projected/f7c00d2e-dc8f-4031-9cd6-1a7a333705cf-kube-api-access-f58fz\") pod \"metrics-server-6c49ff9684-plf67\" (UID: \"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf\") " pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.539164 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.539138    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:18:49.681200 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.681122    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1938e51f-d5c0-4372-8b9e-4519da5b735a" path="/var/lib/kubelet/pods/1938e51f-d5c0-4372-8b9e-4519da5b735a/volumes"
Sep 18 22:18:49.814071 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:49.814034    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c49ff9684-plf67"]
Sep 18 22:18:49.817557 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:18:49.817522    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c00d2e_dc8f_4031_9cd6_1a7a333705cf.slice/crio-24422b5c4a31934207a209c0293f01505d5c8e500f40e82d70995aab2020ea1e WatchSource:0}: Error finding container 24422b5c4a31934207a209c0293f01505d5c8e500f40e82d70995aab2020ea1e: Status 404 returned error can't find the container with id 24422b5c4a31934207a209c0293f01505d5c8e500f40e82d70995aab2020ea1e
Sep 18 22:18:50.160281 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:50.160249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67" event={"ID":"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf","Type":"ContainerStarted","Data":"24422b5c4a31934207a209c0293f01505d5c8e500f40e82d70995aab2020ea1e"}
Sep 18 22:18:52.168748 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:52.168715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67" event={"ID":"f7c00d2e-dc8f-4031-9cd6-1a7a333705cf","Type":"ContainerStarted","Data":"99e39cb144304c96389a06c75724c0f9ada488e72912c957ab8cd5a622a3ca7b"}
Sep 18 22:18:52.193127 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:18:52.193080    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67" podStartSLOduration=1.723389215 podStartE2EDuration="3.193066626s" totalImagesPullingTime="1.469677411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:18:52.192586755 +0000 UTC m=+116.553103615" watchObservedRunningTime="2026-09-18 22:18:52.193066626 +0000 UTC m=+116.553583483"
Sep 18 22:19:07.552652 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:07.552610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:19:07.555045 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:07.555022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f3ceffc-4830-463b-ab1e-8d1880fade2e-metrics-certs\") pod \"network-metrics-daemon-xzzt2\" (UID: \"2f3ceffc-4830-463b-ab1e-8d1880fade2e\") " pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:19:07.799669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:07.799638    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dq7t2\""
Sep 18 22:19:07.807999 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:07.807943    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xzzt2"
Sep 18 22:19:08.087298 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:08.087221    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xzzt2"]
Sep 18 22:19:08.090113 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:19:08.090086    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3ceffc_4830_463b_ab1e_8d1880fade2e.slice/crio-00409c93d43c1939891bb875fa646b2f021f357f582ffc4abdacdd9d60fe32b4 WatchSource:0}: Error finding container 00409c93d43c1939891bb875fa646b2f021f357f582ffc4abdacdd9d60fe32b4: Status 404 returned error can't find the container with id 00409c93d43c1939891bb875fa646b2f021f357f582ffc4abdacdd9d60fe32b4
Sep 18 22:19:08.222809 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:08.222768    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xzzt2" event={"ID":"2f3ceffc-4830-463b-ab1e-8d1880fade2e","Type":"ContainerStarted","Data":"00409c93d43c1939891bb875fa646b2f021f357f582ffc4abdacdd9d60fe32b4"}
Sep 18 22:19:09.233464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:09.229022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xzzt2" event={"ID":"2f3ceffc-4830-463b-ab1e-8d1880fade2e","Type":"ContainerStarted","Data":"d21310122d915b66a7299234d20dd508f6b21bba6f97a4104f730b54edfcd06b"}
Sep 18 22:19:09.233464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:09.229139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xzzt2" event={"ID":"2f3ceffc-4830-463b-ab1e-8d1880fade2e","Type":"ContainerStarted","Data":"f2d673c593b846ba3bd8956e61e068188f9b246f8fca967ccccf8e4afcbe5ac8"}
Sep 18 22:19:09.250347 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:09.250303    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xzzt2" podStartSLOduration=131.398459595 podStartE2EDuration="2m12.250288246s" totalImagesPullingTime="851.828651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:19:09.24892212 +0000 UTC m=+133.609438977" watchObservedRunningTime="2026-09-18 22:19:09.250288246 +0000 UTC m=+133.610805102"
Sep 18 22:19:09.539855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:09.539775    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:19:09.539855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:09.539838    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:19:10.234206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:10.234174    2583 generic.go:356] "Generic (PLEG): container finished" podID="484adadf-5f15-4fe6-b88f-0d6e76b0ae3f" containerID="ae25b43f263e9b7ba0fd035daad8a04f616fa09f74e81b8204f210d2c53fefa5" exitCode=0
Sep 18 22:19:10.234649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:10.234250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv" event={"ID":"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f","Type":"ContainerDied","Data":"ae25b43f263e9b7ba0fd035daad8a04f616fa09f74e81b8204f210d2c53fefa5"}
Sep 18 22:19:10.234760 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:10.234743    2583 scope.go:117] "RemoveContainer" containerID="ae25b43f263e9b7ba0fd035daad8a04f616fa09f74e81b8204f210d2c53fefa5"
Sep 18 22:19:11.239868 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:11.239838    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-p75rv" event={"ID":"484adadf-5f15-4fe6-b88f-0d6e76b0ae3f","Type":"ContainerStarted","Data":"ef313591dba37bece4cb5afeef7e6cee11a00c9b51e8c3d12eaff611ad1903f5"}
Sep 18 22:19:29.545349 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:29.545322    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:19:29.549129 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:19:29.549109    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c49ff9684-plf67"
Sep 18 22:20:09.221745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.221659    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5955dd6588-x2nvs"]
Sep 18 22:20:09.226689 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.225969    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.229222 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.229199    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 22:20:09.229351 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.229292    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 22:20:09.229444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.229359    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v9tgp\""
Sep 18 22:20:09.229444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.229423    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 22:20:09.229557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.229538    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 22:20:09.229640 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.229623    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 22:20:09.234521 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.234500    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 22:20:09.243688 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.243655    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5955dd6588-x2nvs"]
Sep 18 22:20:09.407578 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-secret-telemeter-client\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407578 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-metrics-client-ca\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-telemeter-client-tls\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407680    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407880 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kccl4\" (UniqueName: \"kubernetes.io/projected/35582a02-2a07-4463-a876-45d92cb4deee-kube-api-access-kccl4\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407880 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407803    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-federate-client-tls\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.407880 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.407824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-serving-certs-ca-bundle\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kccl4\" (UniqueName: \"kubernetes.io/projected/35582a02-2a07-4463-a876-45d92cb4deee-kube-api-access-kccl4\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-federate-client-tls\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-serving-certs-ca-bundle\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-secret-telemeter-client\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-metrics-client-ca\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-telemeter-client-tls\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.509525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.509262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.510264 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.510223    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-serving-certs-ca-bundle\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.510264 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.510247    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-metrics-client-ca\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.510450 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.510299    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35582a02-2a07-4463-a876-45d92cb4deee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.511906 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.511885    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.512018 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.511912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-federate-client-tls\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.512212 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.512193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-telemeter-client-tls\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.512274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.512214    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/35582a02-2a07-4463-a876-45d92cb4deee-secret-telemeter-client\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.519108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.519087    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kccl4\" (UniqueName: \"kubernetes.io/projected/35582a02-2a07-4463-a876-45d92cb4deee-kube-api-access-kccl4\") pod \"telemeter-client-5955dd6588-x2nvs\" (UID: \"35582a02-2a07-4463-a876-45d92cb4deee\") " pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.537021 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.536999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs"
Sep 18 22:20:09.814628 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:09.814603    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5955dd6588-x2nvs"]
Sep 18 22:20:09.816502 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:20:09.816478    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35582a02_2a07_4463_a876_45d92cb4deee.slice/crio-7c106567d4aa00720cec9b26f3396947e050295694ef27a8f78889b278b498e8 WatchSource:0}: Error finding container 7c106567d4aa00720cec9b26f3396947e050295694ef27a8f78889b278b498e8: Status 404 returned error can't find the container with id 7c106567d4aa00720cec9b26f3396947e050295694ef27a8f78889b278b498e8
Sep 18 22:20:10.448062 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:10.448023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs" event={"ID":"35582a02-2a07-4463-a876-45d92cb4deee","Type":"ContainerStarted","Data":"7c106567d4aa00720cec9b26f3396947e050295694ef27a8f78889b278b498e8"}
Sep 18 22:20:12.456925 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:12.456887    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs" event={"ID":"35582a02-2a07-4463-a876-45d92cb4deee","Type":"ContainerStarted","Data":"f1bf2feb4bc54fc8e6081538521c4ec698b021ab0bea5073f8c4007b102a1b00"}
Sep 18 22:20:13.462119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:13.462080    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs" event={"ID":"35582a02-2a07-4463-a876-45d92cb4deee","Type":"ContainerStarted","Data":"cd37730dbe88a0745b67bb4a3e00726aa753ce9adb02cb916d061a71c4e7798e"}
Sep 18 22:20:13.462504 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:13.462148    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs" event={"ID":"35582a02-2a07-4463-a876-45d92cb4deee","Type":"ContainerStarted","Data":"0b1c2e8a17896e6731be153ef77086e8e143dd7ab36bd1f5ee2eb13d65c45137"}
Sep 18 22:20:13.491557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:20:13.491510    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5955dd6588-x2nvs" podStartSLOduration=1.39355104 podStartE2EDuration="4.491494897s" totalImagesPullingTime="3.097943857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:20:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:20:13.490618326 +0000 UTC m=+197.851135209" watchObservedRunningTime="2026-09-18 22:20:13.491494897 +0000 UTC m=+197.852011768"
Sep 18 22:21:11.749600 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.749523    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"]
Sep 18 22:21:11.752713 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.752695    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.755166 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.755145    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vzhmn\""
Sep 18 22:21:11.755261 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.755149    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 22:21:11.756051 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.756033    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 22:21:11.765119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.765096    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"]
Sep 18 22:21:11.873852 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.873814    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.874017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.873871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s22sb\" (UniqueName: \"kubernetes.io/projected/091951e9-117c-4ad2-bc8b-a47d2d246a0c-kube-api-access-s22sb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.874017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.873952    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.974852 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.974815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.975032 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.974873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s22sb\" (UniqueName: \"kubernetes.io/projected/091951e9-117c-4ad2-bc8b-a47d2d246a0c-kube-api-access-s22sb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.975032 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.974916    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.975291 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.975271    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.975367 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.975274    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:11.983429 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:11.983407    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s22sb\" (UniqueName: \"kubernetes.io/projected/091951e9-117c-4ad2-bc8b-a47d2d246a0c-kube-api-access-s22sb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:12.062179 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:12.062077    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:12.332932 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:12.332861    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"]
Sep 18 22:21:12.336076 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:21:12.336049    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091951e9_117c_4ad2_bc8b_a47d2d246a0c.slice/crio-ea85c393071b313354a428b9dc06470610c366fd1d40bcc2d89a12effe72d9f3 WatchSource:0}: Error finding container ea85c393071b313354a428b9dc06470610c366fd1d40bcc2d89a12effe72d9f3: Status 404 returned error can't find the container with id ea85c393071b313354a428b9dc06470610c366fd1d40bcc2d89a12effe72d9f3
Sep 18 22:21:12.669444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:12.669408    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5" event={"ID":"091951e9-117c-4ad2-bc8b-a47d2d246a0c","Type":"ContainerStarted","Data":"ea85c393071b313354a428b9dc06470610c366fd1d40bcc2d89a12effe72d9f3"}
Sep 18 22:21:18.695774 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:18.695740    2583 generic.go:356] "Generic (PLEG): container finished" podID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerID="c32d77be6214db95faadbad59bb7958ddb747a5740ecd0a3c3e62961386a872b" exitCode=0
Sep 18 22:21:18.696255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:18.695816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5" event={"ID":"091951e9-117c-4ad2-bc8b-a47d2d246a0c","Type":"ContainerDied","Data":"c32d77be6214db95faadbad59bb7958ddb747a5740ecd0a3c3e62961386a872b"}
Sep 18 22:21:20.704846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:20.704811    2583 generic.go:356] "Generic (PLEG): container finished" podID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerID="626fa9024e4fb558a6cc4a32ed240e614760aad9546492a9b9855a9c345d3ae4" exitCode=0
Sep 18 22:21:20.705221 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:20.704869    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5" event={"ID":"091951e9-117c-4ad2-bc8b-a47d2d246a0c","Type":"ContainerDied","Data":"626fa9024e4fb558a6cc4a32ed240e614760aad9546492a9b9855a9c345d3ae4"}
Sep 18 22:21:27.738636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:27.738599    2583 generic.go:356] "Generic (PLEG): container finished" podID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerID="3bb030c9be63e6fa721e46aed6c43cb5611d88d2e5f2543ceaa6134473aab6d2" exitCode=0
Sep 18 22:21:27.739003 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:27.738643    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5" event={"ID":"091951e9-117c-4ad2-bc8b-a47d2d246a0c","Type":"ContainerDied","Data":"3bb030c9be63e6fa721e46aed6c43cb5611d88d2e5f2543ceaa6134473aab6d2"}
Sep 18 22:21:28.872752 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:28.872728    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:29.013983 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.013895    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s22sb\" (UniqueName: \"kubernetes.io/projected/091951e9-117c-4ad2-bc8b-a47d2d246a0c-kube-api-access-s22sb\") pod \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") "
Sep 18 22:21:29.013983 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.013933    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-bundle\") pod \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") "
Sep 18 22:21:29.014193 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.013989    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-util\") pod \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\" (UID: \"091951e9-117c-4ad2-bc8b-a47d2d246a0c\") "
Sep 18 22:21:29.014626 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.014596    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-bundle" (OuterVolumeSpecName: "bundle") pod "091951e9-117c-4ad2-bc8b-a47d2d246a0c" (UID: "091951e9-117c-4ad2-bc8b-a47d2d246a0c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:21:29.016126 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.016103    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091951e9-117c-4ad2-bc8b-a47d2d246a0c-kube-api-access-s22sb" (OuterVolumeSpecName: "kube-api-access-s22sb") pod "091951e9-117c-4ad2-bc8b-a47d2d246a0c" (UID: "091951e9-117c-4ad2-bc8b-a47d2d246a0c"). InnerVolumeSpecName "kube-api-access-s22sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:21:29.018073 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.018051    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-util" (OuterVolumeSpecName: "util") pod "091951e9-117c-4ad2-bc8b-a47d2d246a0c" (UID: "091951e9-117c-4ad2-bc8b-a47d2d246a0c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:21:29.115371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.115334    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s22sb\" (UniqueName: \"kubernetes.io/projected/091951e9-117c-4ad2-bc8b-a47d2d246a0c-kube-api-access-s22sb\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:21:29.115371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.115364    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:21:29.115371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.115374    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091951e9-117c-4ad2-bc8b-a47d2d246a0c-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:21:29.747303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.747275    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5"
Sep 18 22:21:29.747480 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.747282    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m8n5" event={"ID":"091951e9-117c-4ad2-bc8b-a47d2d246a0c","Type":"ContainerDied","Data":"ea85c393071b313354a428b9dc06470610c366fd1d40bcc2d89a12effe72d9f3"}
Sep 18 22:21:29.747480 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:29.747415    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea85c393071b313354a428b9dc06470610c366fd1d40bcc2d89a12effe72d9f3"
Sep 18 22:21:34.720931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.720898    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"]
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721157    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="util"
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721170    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="util"
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721178    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="extract"
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721184    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="extract"
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721195    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="pull"
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721200    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="pull"
Sep 18 22:21:34.721277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.721238    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="091951e9-117c-4ad2-bc8b-a47d2d246a0c" containerName="extract"
Sep 18 22:21:34.769123 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.769092    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"]
Sep 18 22:21:34.769581 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.769557    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:34.774747 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.774714    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:21:34.774869 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.774776    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-nfrnv\""
Sep 18 22:21:34.775464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.775450    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 18 22:21:34.857798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.857715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b4233e8-ad92-44c8-88df-6e6243fe43ad-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jt98\" (UID: \"5b4233e8-ad92-44c8-88df-6e6243fe43ad\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:34.857798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.857750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qlsl\" (UniqueName: \"kubernetes.io/projected/5b4233e8-ad92-44c8-88df-6e6243fe43ad-kube-api-access-9qlsl\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jt98\" (UID: \"5b4233e8-ad92-44c8-88df-6e6243fe43ad\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:34.958694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.958658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b4233e8-ad92-44c8-88df-6e6243fe43ad-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jt98\" (UID: \"5b4233e8-ad92-44c8-88df-6e6243fe43ad\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:34.958884 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.958701    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qlsl\" (UniqueName: \"kubernetes.io/projected/5b4233e8-ad92-44c8-88df-6e6243fe43ad-kube-api-access-9qlsl\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jt98\" (UID: \"5b4233e8-ad92-44c8-88df-6e6243fe43ad\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:34.959089 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.959068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b4233e8-ad92-44c8-88df-6e6243fe43ad-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jt98\" (UID: \"5b4233e8-ad92-44c8-88df-6e6243fe43ad\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:34.967151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:34.967124    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qlsl\" (UniqueName: \"kubernetes.io/projected/5b4233e8-ad92-44c8-88df-6e6243fe43ad-kube-api-access-9qlsl\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jt98\" (UID: \"5b4233e8-ad92-44c8-88df-6e6243fe43ad\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:35.081988 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:35.081955    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"
Sep 18 22:21:35.379489 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:35.379463    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98"]
Sep 18 22:21:35.381463 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:21:35.381433    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4233e8_ad92_44c8_88df_6e6243fe43ad.slice/crio-87dfc2dcdc1feae585f71ee38154f2112aafb72e628345e1bcdff9cec4803a2b WatchSource:0}: Error finding container 87dfc2dcdc1feae585f71ee38154f2112aafb72e628345e1bcdff9cec4803a2b: Status 404 returned error can't find the container with id 87dfc2dcdc1feae585f71ee38154f2112aafb72e628345e1bcdff9cec4803a2b
Sep 18 22:21:35.769735 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:35.769655    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98" event={"ID":"5b4233e8-ad92-44c8-88df-6e6243fe43ad","Type":"ContainerStarted","Data":"87dfc2dcdc1feae585f71ee38154f2112aafb72e628345e1bcdff9cec4803a2b"}
Sep 18 22:21:37.779867 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:37.779797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98" event={"ID":"5b4233e8-ad92-44c8-88df-6e6243fe43ad","Type":"ContainerStarted","Data":"3af81a1c3b255429b39ba688d9601151fbba0b326b1dead424f7ad0a6e50bc54"}
Sep 18 22:21:37.820802 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:37.820755    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jt98" podStartSLOduration=1.785798473 podStartE2EDuration="3.820737342s" totalImagesPullingTime="2.034938869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:21:37.817266592 +0000 UTC m=+282.177783460" watchObservedRunningTime="2026-09-18 22:21:37.820737342 +0000 UTC m=+282.181254198"
Sep 18 22:21:38.334867 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.334831    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2h24c"]
Sep 18 22:21:38.357248 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.357217    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2h24c"]
Sep 18 22:21:38.357395 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.357333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.360547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.360524    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 22:21:38.361410 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.361375    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 22:21:38.361525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.361432    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jhrmn\""
Sep 18 22:21:38.385580 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.385555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a2613c1-5e78-4204-ba48-db7ce57a9ec4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2h24c\" (UID: \"0a2613c1-5e78-4204-ba48-db7ce57a9ec4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.385671 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.385594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzp5\" (UniqueName: \"kubernetes.io/projected/0a2613c1-5e78-4204-ba48-db7ce57a9ec4-kube-api-access-4vzp5\") pod \"cert-manager-webhook-6486ffd997-2h24c\" (UID: \"0a2613c1-5e78-4204-ba48-db7ce57a9ec4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.486531 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.486498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a2613c1-5e78-4204-ba48-db7ce57a9ec4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2h24c\" (UID: \"0a2613c1-5e78-4204-ba48-db7ce57a9ec4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.486694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.486546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzp5\" (UniqueName: \"kubernetes.io/projected/0a2613c1-5e78-4204-ba48-db7ce57a9ec4-kube-api-access-4vzp5\") pod \"cert-manager-webhook-6486ffd997-2h24c\" (UID: \"0a2613c1-5e78-4204-ba48-db7ce57a9ec4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.501503 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.501468    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzp5\" (UniqueName: \"kubernetes.io/projected/0a2613c1-5e78-4204-ba48-db7ce57a9ec4-kube-api-access-4vzp5\") pod \"cert-manager-webhook-6486ffd997-2h24c\" (UID: \"0a2613c1-5e78-4204-ba48-db7ce57a9ec4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.501913 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.501894    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a2613c1-5e78-4204-ba48-db7ce57a9ec4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2h24c\" (UID: \"0a2613c1-5e78-4204-ba48-db7ce57a9ec4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.666366 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.666333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:38.957887 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:38.957857    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2h24c"]
Sep 18 22:21:38.960497 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:21:38.960468    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a2613c1_5e78_4204_ba48_db7ce57a9ec4.slice/crio-ac68c000e1c6fb2af28a87357431e33ee659842b3f75e790691e3c51901531c5 WatchSource:0}: Error finding container ac68c000e1c6fb2af28a87357431e33ee659842b3f75e790691e3c51901531c5: Status 404 returned error can't find the container with id ac68c000e1c6fb2af28a87357431e33ee659842b3f75e790691e3c51901531c5
Sep 18 22:21:39.025050 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.025013    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"]
Sep 18 22:21:39.055865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.055835    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"]
Sep 18 22:21:39.056034 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.055952    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.058951 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.058926    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vzhmn\""
Sep 18 22:21:39.059090 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.058980    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 22:21:39.059937 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.059917    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 22:21:39.092956 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.092930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.093078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.092967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.093078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.092986    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdqr5\" (UniqueName: \"kubernetes.io/projected/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-kube-api-access-tdqr5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.193708 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.193671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.193881 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.193720    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.193881 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.193748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdqr5\" (UniqueName: \"kubernetes.io/projected/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-kube-api-access-tdqr5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.194043 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.194022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.194093 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.194065    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.202981 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.202958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdqr5\" (UniqueName: \"kubernetes.io/projected/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-kube-api-access-tdqr5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.365275 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.365195    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:39.648422 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.648377    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"]
Sep 18 22:21:39.651319 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:21:39.651282    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a05eab_cc91_4a8b_a24d_c7f930f400e7.slice/crio-366ce3944d96129cca7103cd6b11f70ff360a870e7995affa70f216ef0f5b48b WatchSource:0}: Error finding container 366ce3944d96129cca7103cd6b11f70ff360a870e7995affa70f216ef0f5b48b: Status 404 returned error can't find the container with id 366ce3944d96129cca7103cd6b11f70ff360a870e7995affa70f216ef0f5b48b
Sep 18 22:21:39.795107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.795070    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c" event={"ID":"b8a05eab-cc91-4a8b-a24d-c7f930f400e7","Type":"ContainerStarted","Data":"b4bd37c3b0d4b03bbc9d3c9ab9c0d1cfed9f16f2ef71ec94769bafe30b9a783f"}
Sep 18 22:21:39.795251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.795129    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c" event={"ID":"b8a05eab-cc91-4a8b-a24d-c7f930f400e7","Type":"ContainerStarted","Data":"366ce3944d96129cca7103cd6b11f70ff360a870e7995affa70f216ef0f5b48b"}
Sep 18 22:21:39.796418 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:39.796377    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c" event={"ID":"0a2613c1-5e78-4204-ba48-db7ce57a9ec4","Type":"ContainerStarted","Data":"ac68c000e1c6fb2af28a87357431e33ee659842b3f75e790691e3c51901531c5"}
Sep 18 22:21:40.804088 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:40.804049    2583 generic.go:356] "Generic (PLEG): container finished" podID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerID="b4bd37c3b0d4b03bbc9d3c9ab9c0d1cfed9f16f2ef71ec94769bafe30b9a783f" exitCode=0
Sep 18 22:21:40.804575 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:40.804152    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c" event={"ID":"b8a05eab-cc91-4a8b-a24d-c7f930f400e7","Type":"ContainerDied","Data":"b4bd37c3b0d4b03bbc9d3c9ab9c0d1cfed9f16f2ef71ec94769bafe30b9a783f"}
Sep 18 22:21:41.811405 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:41.811343    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c" event={"ID":"0a2613c1-5e78-4204-ba48-db7ce57a9ec4","Type":"ContainerStarted","Data":"af7a81742b3c5cd937b07fcc5e2234d0d393ac0b3425074b5007bb3a9b61e14c"}
Sep 18 22:21:41.811888 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:41.811448    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:41.827828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:41.827773    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c" podStartSLOduration=1.131458891 podStartE2EDuration="3.827756869s" totalImagesPullingTime="2.696297978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:21:41.8269249 +0000 UTC m=+286.187441758" watchObservedRunningTime="2026-09-18 22:21:41.827756869 +0000 UTC m=+286.188273725"
Sep 18 22:21:43.822365 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:43.822327    2583 generic.go:356] "Generic (PLEG): container finished" podID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerID="fc4cca000a94b22bba59699d448e9b963b4552c466319cd970fdb00bad95fa0a" exitCode=0
Sep 18 22:21:43.822773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:43.822419    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c" event={"ID":"b8a05eab-cc91-4a8b-a24d-c7f930f400e7","Type":"ContainerDied","Data":"fc4cca000a94b22bba59699d448e9b963b4552c466319cd970fdb00bad95fa0a"}
Sep 18 22:21:44.829280 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:44.829249    2583 generic.go:356] "Generic (PLEG): container finished" podID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerID="fb762058c78176e2a5d11e203607212cbcd87215accee854aba46a1e75c67f85" exitCode=0
Sep 18 22:21:44.829686 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:44.829341    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c" event={"ID":"b8a05eab-cc91-4a8b-a24d-c7f930f400e7","Type":"ContainerDied","Data":"fb762058c78176e2a5d11e203607212cbcd87215accee854aba46a1e75c67f85"}
Sep 18 22:21:45.956775 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:45.956748    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:46.049182 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.049146    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-bundle\") pod \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") "
Sep 18 22:21:46.049182 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.049189    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-util\") pod \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") "
Sep 18 22:21:46.049371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.049240    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdqr5\" (UniqueName: \"kubernetes.io/projected/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-kube-api-access-tdqr5\") pod \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\" (UID: \"b8a05eab-cc91-4a8b-a24d-c7f930f400e7\") "
Sep 18 22:21:46.049640 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.049614    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-bundle" (OuterVolumeSpecName: "bundle") pod "b8a05eab-cc91-4a8b-a24d-c7f930f400e7" (UID: "b8a05eab-cc91-4a8b-a24d-c7f930f400e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:21:46.051268 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.051248    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-kube-api-access-tdqr5" (OuterVolumeSpecName: "kube-api-access-tdqr5") pod "b8a05eab-cc91-4a8b-a24d-c7f930f400e7" (UID: "b8a05eab-cc91-4a8b-a24d-c7f930f400e7"). InnerVolumeSpecName "kube-api-access-tdqr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:21:46.054984 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.054960    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-util" (OuterVolumeSpecName: "util") pod "b8a05eab-cc91-4a8b-a24d-c7f930f400e7" (UID: "b8a05eab-cc91-4a8b-a24d-c7f930f400e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:21:46.150004 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.149976    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdqr5\" (UniqueName: \"kubernetes.io/projected/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-kube-api-access-tdqr5\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:21:46.150004 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.150000    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:21:46.150004 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.150009    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8a05eab-cc91-4a8b-a24d-c7f930f400e7-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:21:46.839009 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.838979    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c"
Sep 18 22:21:46.839009 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.838999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqwh5c" event={"ID":"b8a05eab-cc91-4a8b-a24d-c7f930f400e7","Type":"ContainerDied","Data":"366ce3944d96129cca7103cd6b11f70ff360a870e7995affa70f216ef0f5b48b"}
Sep 18 22:21:46.839198 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:46.839048    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="366ce3944d96129cca7103cd6b11f70ff360a870e7995affa70f216ef0f5b48b"
Sep 18 22:21:47.818973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:47.818944    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2h24c"
Sep 18 22:21:48.948691 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948655    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-khkqb"]
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948918    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="pull"
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948931    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="pull"
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948944    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="extract"
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948949    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="extract"
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948967    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="util"
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.948973    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="util"
Sep 18 22:21:48.949078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.949016    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8a05eab-cc91-4a8b-a24d-c7f930f400e7" containerName="extract"
Sep 18 22:21:48.952974 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.952959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:48.955380 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.955349    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9mm6r\""
Sep 18 22:21:48.961585 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:48.961559    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-khkqb"]
Sep 18 22:21:49.071884 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.071852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lfs9\" (UniqueName: \"kubernetes.io/projected/5952de86-9326-446b-b98e-5161bf8476f2-kube-api-access-7lfs9\") pod \"cert-manager-55b4d954d5-khkqb\" (UID: \"5952de86-9326-446b-b98e-5161bf8476f2\") " pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.072025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.071898    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5952de86-9326-446b-b98e-5161bf8476f2-bound-sa-token\") pod \"cert-manager-55b4d954d5-khkqb\" (UID: \"5952de86-9326-446b-b98e-5161bf8476f2\") " pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.173140 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.173105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lfs9\" (UniqueName: \"kubernetes.io/projected/5952de86-9326-446b-b98e-5161bf8476f2-kube-api-access-7lfs9\") pod \"cert-manager-55b4d954d5-khkqb\" (UID: \"5952de86-9326-446b-b98e-5161bf8476f2\") " pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.173140 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.173141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5952de86-9326-446b-b98e-5161bf8476f2-bound-sa-token\") pod \"cert-manager-55b4d954d5-khkqb\" (UID: \"5952de86-9326-446b-b98e-5161bf8476f2\") " pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.181440 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.181410    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5952de86-9326-446b-b98e-5161bf8476f2-bound-sa-token\") pod \"cert-manager-55b4d954d5-khkqb\" (UID: \"5952de86-9326-446b-b98e-5161bf8476f2\") " pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.181737 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.181715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lfs9\" (UniqueName: \"kubernetes.io/projected/5952de86-9326-446b-b98e-5161bf8476f2-kube-api-access-7lfs9\") pod \"cert-manager-55b4d954d5-khkqb\" (UID: \"5952de86-9326-446b-b98e-5161bf8476f2\") " pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.261741 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.261662    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-khkqb"
Sep 18 22:21:49.533979 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.533899    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-khkqb"]
Sep 18 22:21:49.537427 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:21:49.537379    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5952de86_9326_446b_b98e_5161bf8476f2.slice/crio-f4728a5bfe06cc0d0feb9c23c989da473286379d99d1ea0fe88eb2d73741807b WatchSource:0}: Error finding container f4728a5bfe06cc0d0feb9c23c989da473286379d99d1ea0fe88eb2d73741807b: Status 404 returned error can't find the container with id f4728a5bfe06cc0d0feb9c23c989da473286379d99d1ea0fe88eb2d73741807b
Sep 18 22:21:49.852827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.852742    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-khkqb" event={"ID":"5952de86-9326-446b-b98e-5161bf8476f2","Type":"ContainerStarted","Data":"80641510daad15002369e96e712c43211cad0d1a24884d9f3da316dc9364fe26"}
Sep 18 22:21:49.852827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.852782    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-khkqb" event={"ID":"5952de86-9326-446b-b98e-5161bf8476f2","Type":"ContainerStarted","Data":"f4728a5bfe06cc0d0feb9c23c989da473286379d99d1ea0fe88eb2d73741807b"}
Sep 18 22:21:49.869629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:49.869582    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-khkqb" podStartSLOduration=1.8695683060000001 podStartE2EDuration="1.869568306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:21:49.86941712 +0000 UTC m=+294.229933981" watchObservedRunningTime="2026-09-18 22:21:49.869568306 +0000 UTC m=+294.230085162"
Sep 18 22:21:57.550691 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:57.550648    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:21:57.551275 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:57.551258    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:21:57.560554 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:21:57.560536    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 22:22:04.370537 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.370504    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"]
Sep 18 22:22:04.373807 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.373793    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.376443 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.376419    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 22:22:04.376553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.376419    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 22:22:04.377262 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.377247    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vzhmn\""
Sep 18 22:22:04.384410 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.384371    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"]
Sep 18 22:22:04.491795 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.491765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.491962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.491800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.491962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.491831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lfgr\" (UniqueName: \"kubernetes.io/projected/cac75a26-7fe1-4c1c-9973-01fe266de4c1-kube-api-access-9lfgr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.592870 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.592839    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.592870 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.592873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.592998 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.592908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lfgr\" (UniqueName: \"kubernetes.io/projected/cac75a26-7fe1-4c1c-9973-01fe266de4c1-kube-api-access-9lfgr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.593208 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.593190    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.593247 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.593219    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.601715 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.601685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lfgr\" (UniqueName: \"kubernetes.io/projected/cac75a26-7fe1-4c1c-9973-01fe266de4c1-kube-api-access-9lfgr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.682812 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.682778    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:04.952967 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:04.952948    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"]
Sep 18 22:22:04.955077 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:04.955043    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcac75a26_7fe1_4c1c_9973_01fe266de4c1.slice/crio-640375e6ef6b87ca4d7045f8f2c2d2ae77e036fe02c3b51d764762c275439151 WatchSource:0}: Error finding container 640375e6ef6b87ca4d7045f8f2c2d2ae77e036fe02c3b51d764762c275439151: Status 404 returned error can't find the container with id 640375e6ef6b87ca4d7045f8f2c2d2ae77e036fe02c3b51d764762c275439151
Sep 18 22:22:05.918830 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:05.918796    2583 generic.go:356] "Generic (PLEG): container finished" podID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerID="8c861dc289eac8347b7962bdc7d0f578110430b5dbb6dba7ae807e5b1ab2d901" exitCode=0
Sep 18 22:22:05.919251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:05.918880    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w" event={"ID":"cac75a26-7fe1-4c1c-9973-01fe266de4c1","Type":"ContainerDied","Data":"8c861dc289eac8347b7962bdc7d0f578110430b5dbb6dba7ae807e5b1ab2d901"}
Sep 18 22:22:05.919251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:05.918928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w" event={"ID":"cac75a26-7fe1-4c1c-9973-01fe266de4c1","Type":"ContainerStarted","Data":"640375e6ef6b87ca4d7045f8f2c2d2ae77e036fe02c3b51d764762c275439151"}
Sep 18 22:22:05.919327 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:05.919288    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:22:06.925614 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:06.925581    2583 generic.go:356] "Generic (PLEG): container finished" podID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerID="9a31ac5bd941aeb85116c06377a420d8faf871a1b5a607e1a2cb75ba35256860" exitCode=0
Sep 18 22:22:06.926025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:06.925632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w" event={"ID":"cac75a26-7fe1-4c1c-9973-01fe266de4c1","Type":"ContainerDied","Data":"9a31ac5bd941aeb85116c06377a420d8faf871a1b5a607e1a2cb75ba35256860"}
Sep 18 22:22:07.935245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:07.935210    2583 generic.go:356] "Generic (PLEG): container finished" podID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerID="4b84532b73aa88b484b41ee6934b4fe90c604fe0ac9506824eec7613bc1978e4" exitCode=0
Sep 18 22:22:07.935621 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:07.935298    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w" event={"ID":"cac75a26-7fe1-4c1c-9973-01fe266de4c1","Type":"ContainerDied","Data":"4b84532b73aa88b484b41ee6934b4fe90c604fe0ac9506824eec7613bc1978e4"}
Sep 18 22:22:09.067017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.066994    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:09.128338 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.128306    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-bundle\") pod \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") "
Sep 18 22:22:09.128338 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.128342    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lfgr\" (UniqueName: \"kubernetes.io/projected/cac75a26-7fe1-4c1c-9973-01fe266de4c1-kube-api-access-9lfgr\") pod \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") "
Sep 18 22:22:09.128514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.128404    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-util\") pod \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\" (UID: \"cac75a26-7fe1-4c1c-9973-01fe266de4c1\") "
Sep 18 22:22:09.129059 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.129031    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-bundle" (OuterVolumeSpecName: "bundle") pod "cac75a26-7fe1-4c1c-9973-01fe266de4c1" (UID: "cac75a26-7fe1-4c1c-9973-01fe266de4c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:22:09.130527 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.130504    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cac75a26-7fe1-4c1c-9973-01fe266de4c1-kube-api-access-9lfgr" (OuterVolumeSpecName: "kube-api-access-9lfgr") pod "cac75a26-7fe1-4c1c-9973-01fe266de4c1" (UID: "cac75a26-7fe1-4c1c-9973-01fe266de4c1"). InnerVolumeSpecName "kube-api-access-9lfgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:22:09.134196 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.134176    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-util" (OuterVolumeSpecName: "util") pod "cac75a26-7fe1-4c1c-9973-01fe266de4c1" (UID: "cac75a26-7fe1-4c1c-9973-01fe266de4c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:22:09.229898 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.229816    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:22:09.229898 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.229844    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac75a26-7fe1-4c1c-9973-01fe266de4c1-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:22:09.229898 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.229853    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lfgr\" (UniqueName: \"kubernetes.io/projected/cac75a26-7fe1-4c1c-9973-01fe266de4c1-kube-api-access-9lfgr\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:22:09.709035 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709005    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"]
Sep 18 22:22:09.709326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709311    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="util"
Sep 18 22:22:09.709430 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709330    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="util"
Sep 18 22:22:09.709430 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709346    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="extract"
Sep 18 22:22:09.709430 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709354    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="extract"
Sep 18 22:22:09.709430 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709369    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="pull"
Sep 18 22:22:09.709430 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709377    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="pull"
Sep 18 22:22:09.709702 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.709465    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cac75a26-7fe1-4c1c-9973-01fe266de4c1" containerName="extract"
Sep 18 22:22:09.712583 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.712564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.715151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.715132    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 22:22:09.716504 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.716480    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rrh9w\""
Sep 18 22:22:09.720180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.716713    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 22:22:09.720180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.716808    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 22:22:09.720180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.716723    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:22:09.720180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.716952    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 22:22:09.725614 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.725590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"]
Sep 18 22:22:09.834185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.834156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1a354c3-d392-49e6-845d-d18129134b5c-metrics-cert\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.834185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.834189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1a354c3-d392-49e6-845d-d18129134b5c-manager-config\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.834381 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.834212    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a354c3-d392-49e6-845d-d18129134b5c-cert\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.834381 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.834340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjdn4\" (UniqueName: \"kubernetes.io/projected/d1a354c3-d392-49e6-845d-d18129134b5c-kube-api-access-sjdn4\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.935497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.935463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a354c3-d392-49e6-845d-d18129134b5c-cert\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.935634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.935539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjdn4\" (UniqueName: \"kubernetes.io/projected/d1a354c3-d392-49e6-845d-d18129134b5c-kube-api-access-sjdn4\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.935634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.935591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1a354c3-d392-49e6-845d-d18129134b5c-metrics-cert\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.935634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.935621    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1a354c3-d392-49e6-845d-d18129134b5c-manager-config\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.936266 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.936248    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1a354c3-d392-49e6-845d-d18129134b5c-manager-config\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.938003 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.937975    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1a354c3-d392-49e6-845d-d18129134b5c-metrics-cert\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.938141 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.938034    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a354c3-d392-49e6-845d-d18129134b5c-cert\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:09.946099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.946054    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w"
Sep 18 22:22:09.946099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.946078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2x86w" event={"ID":"cac75a26-7fe1-4c1c-9973-01fe266de4c1","Type":"ContainerDied","Data":"640375e6ef6b87ca4d7045f8f2c2d2ae77e036fe02c3b51d764762c275439151"}
Sep 18 22:22:09.946209 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.946130    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="640375e6ef6b87ca4d7045f8f2c2d2ae77e036fe02c3b51d764762c275439151"
Sep 18 22:22:09.946279 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:09.946264    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjdn4\" (UniqueName: \"kubernetes.io/projected/d1a354c3-d392-49e6-845d-d18129134b5c-kube-api-access-sjdn4\") pod \"lws-controller-manager-5bdd6569c9-bpbw4\" (UID: \"d1a354c3-d392-49e6-845d-d18129134b5c\") " pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:10.023678 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:10.023644    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:10.295846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:10.295775    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"]
Sep 18 22:22:10.298633 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:10.298607    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a354c3_d392_49e6_845d_d18129134b5c.slice/crio-54ca6316c68ec89b31e1468bf509a4bbb16b76675ce263a11644c13a4a0fcee4 WatchSource:0}: Error finding container 54ca6316c68ec89b31e1468bf509a4bbb16b76675ce263a11644c13a4a0fcee4: Status 404 returned error can't find the container with id 54ca6316c68ec89b31e1468bf509a4bbb16b76675ce263a11644c13a4a0fcee4
Sep 18 22:22:10.951069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:10.951037    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4" event={"ID":"d1a354c3-d392-49e6-845d-d18129134b5c","Type":"ContainerStarted","Data":"54ca6316c68ec89b31e1468bf509a4bbb16b76675ce263a11644c13a4a0fcee4"}
Sep 18 22:22:12.961762 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:12.961726    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4" event={"ID":"d1a354c3-d392-49e6-845d-d18129134b5c","Type":"ContainerStarted","Data":"bfb38e9e20e68104d790f95593a0726dd2a942527fba2da7183cafd98d82fb47"}
Sep 18 22:22:12.962192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:12.961782    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:12.979705 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:12.979654    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4" podStartSLOduration=2.298696258 podStartE2EDuration="3.979637114s" totalImagesPullingTime="1.680940856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:22:12.978512631 +0000 UTC m=+317.339029500" watchObservedRunningTime="2026-09-18 22:22:12.979637114 +0000 UTC m=+317.340153970"
Sep 18 22:22:14.802902 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.802868    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"]
Sep 18 22:22:14.808312 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.808294    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:14.812063 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.812038    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 22:22:14.812192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.812096    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vzhmn\""
Sep 18 22:22:14.813084 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.813063    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 22:22:14.819625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.819601    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"]
Sep 18 22:22:14.978441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.978412    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:14.978619 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.978452    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22vr7\" (UniqueName: \"kubernetes.io/projected/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-kube-api-access-22vr7\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:14.978619 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:14.978474    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.079524 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.079444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.079524 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.079491    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22vr7\" (UniqueName: \"kubernetes.io/projected/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-kube-api-access-22vr7\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.079524 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.079512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.079842 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.079819    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.079892 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.079876    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.088275 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.088246    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22vr7\" (UniqueName: \"kubernetes.io/projected/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-kube-api-access-22vr7\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.116865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.116843    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:15.386875 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.386845    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"]
Sep 18 22:22:15.390460 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:15.390429    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb1cd7c_0fb7_4626_b1fc_5b36286850e9.slice/crio-f51214e0ac6e84fa7ecca5482f6594db276c6669ed6f97a8543069c6df3b3006 WatchSource:0}: Error finding container f51214e0ac6e84fa7ecca5482f6594db276c6669ed6f97a8543069c6df3b3006: Status 404 returned error can't find the container with id f51214e0ac6e84fa7ecca5482f6594db276c6669ed6f97a8543069c6df3b3006
Sep 18 22:22:15.976014 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.975971    2583 generic.go:356] "Generic (PLEG): container finished" podID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerID="a7942421f690c710e52b6f64573eaf45fcb8fbd5d7a0536c2f6e13e8b07c6b2a" exitCode=0
Sep 18 22:22:15.976421 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.976059    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g" event={"ID":"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9","Type":"ContainerDied","Data":"a7942421f690c710e52b6f64573eaf45fcb8fbd5d7a0536c2f6e13e8b07c6b2a"}
Sep 18 22:22:15.976421 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:15.976111    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g" event={"ID":"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9","Type":"ContainerStarted","Data":"f51214e0ac6e84fa7ecca5482f6594db276c6669ed6f97a8543069c6df3b3006"}
Sep 18 22:22:16.982099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:16.982068    2583 generic.go:356] "Generic (PLEG): container finished" podID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerID="2bf2cddfad60a1770d1b7c3c40cd3112529c75e6214b346beaf40a372b53f8fe" exitCode=0
Sep 18 22:22:16.982492 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:16.982145    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g" event={"ID":"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9","Type":"ContainerDied","Data":"2bf2cddfad60a1770d1b7c3c40cd3112529c75e6214b346beaf40a372b53f8fe"}
Sep 18 22:22:17.384260 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.384230    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"]
Sep 18 22:22:17.387439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.387414    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.392451 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.392427    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 22:22:17.392620 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.392604    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 22:22:17.393124 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.393101    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 22:22:17.393217 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.393139    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 22:22:17.393818 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.393802    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6vdvw\""
Sep 18 22:22:17.407420 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.407396    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"]
Sep 18 22:22:17.497231 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.497196    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83ceed36-4803-45b4-91d6-f1db76358bf0-tmp\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.497368 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.497250    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83ceed36-4803-45b4-91d6-f1db76358bf0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.497368 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.497316    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83ceed36-4803-45b4-91d6-f1db76358bf0-manifests\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.497368 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.497335    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wmz8\" (UniqueName: \"kubernetes.io/projected/83ceed36-4803-45b4-91d6-f1db76358bf0-kube-api-access-9wmz8\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.497502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.497407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83ceed36-4803-45b4-91d6-f1db76358bf0-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.598161 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.598133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83ceed36-4803-45b4-91d6-f1db76358bf0-tmp\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.598254 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.598183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83ceed36-4803-45b4-91d6-f1db76358bf0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.598254 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.598221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83ceed36-4803-45b4-91d6-f1db76358bf0-manifests\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.598254 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.598239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmz8\" (UniqueName: \"kubernetes.io/projected/83ceed36-4803-45b4-91d6-f1db76358bf0-kube-api-access-9wmz8\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.598369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.598256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83ceed36-4803-45b4-91d6-f1db76358bf0-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.598657 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.598631    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/83ceed36-4803-45b4-91d6-f1db76358bf0-manifests\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.600399 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.600356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83ceed36-4803-45b4-91d6-f1db76358bf0-tmp\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.600628 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.600613    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/83ceed36-4803-45b4-91d6-f1db76358bf0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.600706 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.600688    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/83ceed36-4803-45b4-91d6-f1db76358bf0-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.607355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.607336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wmz8\" (UniqueName: \"kubernetes.io/projected/83ceed36-4803-45b4-91d6-f1db76358bf0-kube-api-access-9wmz8\") pod \"opendatahub-operator-controller-manager-65c85dbc8b-z9cgl\" (UID: \"83ceed36-4803-45b4-91d6-f1db76358bf0\") " pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.697852 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.697764    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:17.981722 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.981642    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"]
Sep 18 22:22:17.985197 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:17.985172    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ceed36_4803_45b4_91d6_f1db76358bf0.slice/crio-d8570a0cd43066895c9df3d46266caa1675b1fa3bbe7f2d8a36887963e3a46d7 WatchSource:0}: Error finding container d8570a0cd43066895c9df3d46266caa1675b1fa3bbe7f2d8a36887963e3a46d7: Status 404 returned error can't find the container with id d8570a0cd43066895c9df3d46266caa1675b1fa3bbe7f2d8a36887963e3a46d7
Sep 18 22:22:17.995223 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.995199    2583 generic.go:356] "Generic (PLEG): container finished" podID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerID="2d2ffe8f08235002a11393adb869f82ca74d402d57b0052dd874f45577f173b0" exitCode=0
Sep 18 22:22:17.995303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:17.995241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g" event={"ID":"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9","Type":"ContainerDied","Data":"2d2ffe8f08235002a11393adb869f82ca74d402d57b0052dd874f45577f173b0"}
Sep 18 22:22:19.003024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.002984    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl" event={"ID":"83ceed36-4803-45b4-91d6-f1db76358bf0","Type":"ContainerStarted","Data":"d8570a0cd43066895c9df3d46266caa1675b1fa3bbe7f2d8a36887963e3a46d7"}
Sep 18 22:22:19.158989 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.158952    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:19.314958 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.314881    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-util\") pod \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") "
Sep 18 22:22:19.314958 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.314937    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-bundle\") pod \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") "
Sep 18 22:22:19.315236 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.314998    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22vr7\" (UniqueName: \"kubernetes.io/projected/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-kube-api-access-22vr7\") pod \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\" (UID: \"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9\") "
Sep 18 22:22:19.317339 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.317306    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-bundle" (OuterVolumeSpecName: "bundle") pod "6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" (UID: "6cb1cd7c-0fb7-4626-b1fc-5b36286850e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:22:19.317533 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.317378    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-kube-api-access-22vr7" (OuterVolumeSpecName: "kube-api-access-22vr7") pod "6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" (UID: "6cb1cd7c-0fb7-4626-b1fc-5b36286850e9"). InnerVolumeSpecName "kube-api-access-22vr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:22:19.322554 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.322521    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-util" (OuterVolumeSpecName: "util") pod "6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" (UID: "6cb1cd7c-0fb7-4626-b1fc-5b36286850e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:22:19.415860 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.415821    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:22:19.415860 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.415861    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:22:19.416163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:19.415874    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22vr7\" (UniqueName: \"kubernetes.io/projected/6cb1cd7c-0fb7-4626-b1fc-5b36286850e9-kube-api-access-22vr7\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:22:20.010037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:20.009963    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g"
Sep 18 22:22:20.010037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:20.009977    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0xgv2g" event={"ID":"6cb1cd7c-0fb7-4626-b1fc-5b36286850e9","Type":"ContainerDied","Data":"f51214e0ac6e84fa7ecca5482f6594db276c6669ed6f97a8543069c6df3b3006"}
Sep 18 22:22:20.010037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:20.010019    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f51214e0ac6e84fa7ecca5482f6594db276c6669ed6f97a8543069c6df3b3006"
Sep 18 22:22:21.016660 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:21.016629    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl" event={"ID":"83ceed36-4803-45b4-91d6-f1db76358bf0","Type":"ContainerStarted","Data":"e8a6ec92a82c86944451e27556c8dbf40559fdcc6ed6f4204d075cde4a3066a8"}
Sep 18 22:22:23.029036 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:23.028998    2583 generic.go:356] "Generic (PLEG): container finished" podID="83ceed36-4803-45b4-91d6-f1db76358bf0" containerID="e8a6ec92a82c86944451e27556c8dbf40559fdcc6ed6f4204d075cde4a3066a8" exitCode=0
Sep 18 22:22:23.029411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:23.029074    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl" event={"ID":"83ceed36-4803-45b4-91d6-f1db76358bf0","Type":"ContainerDied","Data":"e8a6ec92a82c86944451e27556c8dbf40559fdcc6ed6f4204d075cde4a3066a8"}
Sep 18 22:22:23.968355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:23.968326    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bdd6569c9-bpbw4"
Sep 18 22:22:24.036110 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:24.036078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl" event={"ID":"83ceed36-4803-45b4-91d6-f1db76358bf0","Type":"ContainerStarted","Data":"4043118831e3d8fdb3238a892b44636996633e5cc6945532d3d5574976030b0c"}
Sep 18 22:22:24.036481 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:24.036292    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:24.059838 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:24.059788    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl" podStartSLOduration=2.648152711 podStartE2EDuration="7.059773433s" totalImagesPullingTime="3.386094805s" totalInitContainerRuntime="1.025525917s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:22:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:22:24.058274973 +0000 UTC m=+328.418791847" watchObservedRunningTime="2026-09-18 22:22:24.059773433 +0000 UTC m=+328.420290289"
Sep 18 22:22:35.042749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:35.042718    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65c85dbc8b-z9cgl"
Sep 18 22:22:47.491002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.490902    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-89gq7"]
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491203    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="util"
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491214    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="util"
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491235    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="extract"
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491240    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="extract"
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491249    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="pull"
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491255    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="pull"
Sep 18 22:22:47.491502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.491302    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb1cd7c-0fb7-4626-b1fc-5b36286850e9" containerName="extract"
Sep 18 22:22:47.494428 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.494412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.496917 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.496889    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xd66v\""
Sep 18 22:22:47.496917 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.496895    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 22:22:47.505164 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.505142    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-89gq7"]
Sep 18 22:22:47.537910 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.537873    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldscd\" (UniqueName: \"kubernetes.io/projected/8e117632-92ea-4ec1-9b93-5b98e0088601-kube-api-access-ldscd\") pod \"odh-model-controller-7998c67dc4-89gq7\" (UID: \"8e117632-92ea-4ec1-9b93-5b98e0088601\") " pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.538078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.537935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e117632-92ea-4ec1-9b93-5b98e0088601-cert\") pod \"odh-model-controller-7998c67dc4-89gq7\" (UID: \"8e117632-92ea-4ec1-9b93-5b98e0088601\") " pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.639101 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.639067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldscd\" (UniqueName: \"kubernetes.io/projected/8e117632-92ea-4ec1-9b93-5b98e0088601-kube-api-access-ldscd\") pod \"odh-model-controller-7998c67dc4-89gq7\" (UID: \"8e117632-92ea-4ec1-9b93-5b98e0088601\") " pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.639290 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.639119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e117632-92ea-4ec1-9b93-5b98e0088601-cert\") pod \"odh-model-controller-7998c67dc4-89gq7\" (UID: \"8e117632-92ea-4ec1-9b93-5b98e0088601\") " pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.641635 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.641615    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e117632-92ea-4ec1-9b93-5b98e0088601-cert\") pod \"odh-model-controller-7998c67dc4-89gq7\" (UID: \"8e117632-92ea-4ec1-9b93-5b98e0088601\") " pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.647586 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.647565    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldscd\" (UniqueName: \"kubernetes.io/projected/8e117632-92ea-4ec1-9b93-5b98e0088601-kube-api-access-ldscd\") pod \"odh-model-controller-7998c67dc4-89gq7\" (UID: \"8e117632-92ea-4ec1-9b93-5b98e0088601\") " pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.806730 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.806646    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:47.809551 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.809521    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m6sss"]
Sep 18 22:22:47.812980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.812957    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:47.815669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.815648    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 22:22:47.824541 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.824513    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m6sss"]
Sep 18 22:22:47.839840 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.839814    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xvpz\" (UniqueName: \"kubernetes.io/projected/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-kube-api-access-7xvpz\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:47.840011 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.839869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:47.941097 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.941060    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xvpz\" (UniqueName: \"kubernetes.io/projected/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-kube-api-access-7xvpz\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:47.941304 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.941127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:47.941304 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:22:47.941250    2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 18 22:22:47.941467 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:22:47.941324    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-tls-certs podName:88b74bf2-c493-4c1b-8532-fcbf9671a6e1 nodeName:}" failed. No retries permitted until 2026-09-18 22:22:48.44130224 +0000 UTC m=+352.801819076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-tls-certs") pod "model-serving-api-54d89bd5fc-m6sss" (UID: "88b74bf2-c493-4c1b-8532-fcbf9671a6e1") : secret "model-serving-api-tls" not found
Sep 18 22:22:47.952154 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:47.952132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xvpz\" (UniqueName: \"kubernetes.io/projected/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-kube-api-access-7xvpz\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:48.079563 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:48.079407    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-89gq7"]
Sep 18 22:22:48.081768 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:48.081741    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e117632_92ea_4ec1_9b93_5b98e0088601.slice/crio-97facf9d49232a61a0d652e22b7ea16775c698eeaa87e66e7ec7feb1fde218f1 WatchSource:0}: Error finding container 97facf9d49232a61a0d652e22b7ea16775c698eeaa87e66e7ec7feb1fde218f1: Status 404 returned error can't find the container with id 97facf9d49232a61a0d652e22b7ea16775c698eeaa87e66e7ec7feb1fde218f1
Sep 18 22:22:48.145032 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:48.144994    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7" event={"ID":"8e117632-92ea-4ec1-9b93-5b98e0088601","Type":"ContainerStarted","Data":"97facf9d49232a61a0d652e22b7ea16775c698eeaa87e66e7ec7feb1fde218f1"}
Sep 18 22:22:48.446698 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:48.446659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:48.449106 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:48.449078    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88b74bf2-c493-4c1b-8532-fcbf9671a6e1-tls-certs\") pod \"model-serving-api-54d89bd5fc-m6sss\" (UID: \"88b74bf2-c493-4c1b-8532-fcbf9671a6e1\") " pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:48.738445 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:48.738348    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:49.028728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:49.028702    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m6sss"]
Sep 18 22:22:49.031150 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:49.031108    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b74bf2_c493_4c1b_8532_fcbf9671a6e1.slice/crio-8cda36ec230ae41b804d89570bfaa6b83c1160730bc843a20a539d3da28762cf WatchSource:0}: Error finding container 8cda36ec230ae41b804d89570bfaa6b83c1160730bc843a20a539d3da28762cf: Status 404 returned error can't find the container with id 8cda36ec230ae41b804d89570bfaa6b83c1160730bc843a20a539d3da28762cf
Sep 18 22:22:49.151572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:49.151533    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss" event={"ID":"88b74bf2-c493-4c1b-8532-fcbf9671a6e1","Type":"ContainerStarted","Data":"8cda36ec230ae41b804d89570bfaa6b83c1160730bc843a20a539d3da28762cf"}
Sep 18 22:22:51.168152 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:51.168095    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7" event={"ID":"8e117632-92ea-4ec1-9b93-5b98e0088601","Type":"ContainerStarted","Data":"c1412e587d7a7c9cd8387a92b3ca2018f193555013414e03c998f3ece1568219"}
Sep 18 22:22:51.168702 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:51.168201    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:22:53.180058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:53.180026    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss" event={"ID":"88b74bf2-c493-4c1b-8532-fcbf9671a6e1","Type":"ContainerStarted","Data":"21c1168f658e359226b03b76fa17a267c0821483b52af06df7323e56a1362aac"}
Sep 18 22:22:53.180539 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:53.180106    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:22:53.197944 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:53.197881    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss" podStartSLOduration=2.566090163 podStartE2EDuration="6.197865885s" totalImagesPullingTime="3.631775722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:22:53.197244667 +0000 UTC m=+357.557761524" watchObservedRunningTime="2026-09-18 22:22:53.197865885 +0000 UTC m=+357.558382743"
Sep 18 22:22:53.198226 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:53.198074    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7" podStartSLOduration=3.451699154 podStartE2EDuration="6.198061801s" totalImagesPullingTime="2.746362647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:22:51.187236728 +0000 UTC m=+355.547753587" watchObservedRunningTime="2026-09-18 22:22:53.198061801 +0000 UTC m=+357.558578658"
Sep 18 22:22:54.556828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.556791    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"]
Sep 18 22:22:54.560034 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.560017    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.564881 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.564858    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 22:22:54.565275 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.565260    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4t99f\""
Sep 18 22:22:54.565448 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.565435    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 22:22:54.565854 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.565830    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 22:22:54.565985 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.565967    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 22:22:54.570224 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.570206    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 22:22:54.572490 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.572476    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 22:22:54.598931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.598903    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"]
Sep 18 22:22:54.599243 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599220    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.599292 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.599326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.599363 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599329    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.599454 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599438    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.599553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599539    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbbzg\" (UniqueName: \"kubernetes.io/projected/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-kube-api-access-xbbzg\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.599590 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.599573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700145 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbbzg\" (UniqueName: \"kubernetes.io/projected/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-kube-api-access-xbbzg\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700145 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700421 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700421 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700354    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700420    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.700881 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.700849    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.702717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.702694    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.702814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.702716    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.702814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.702695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.702814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.702735    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.718737 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.718713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbbzg\" (UniqueName: \"kubernetes.io/projected/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-kube-api-access-xbbzg\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.721661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.721639    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eef8e44b-6cd2-4943-b7f2-0444ebc9452d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-z44wc\" (UID: \"eef8e44b-6cd2-4943-b7f2-0444ebc9452d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:54.869622 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:54.869544    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:55.148682 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:55.148652    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"]
Sep 18 22:22:55.152163 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:22:55.152134    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef8e44b_6cd2_4943_b7f2_0444ebc9452d.slice/crio-5064d033e9286084f5eacbb74ef298e4cded486eb3df961bfffe059b2abd5e06 WatchSource:0}: Error finding container 5064d033e9286084f5eacbb74ef298e4cded486eb3df961bfffe059b2abd5e06: Status 404 returned error can't find the container with id 5064d033e9286084f5eacbb74ef298e4cded486eb3df961bfffe059b2abd5e06
Sep 18 22:22:55.189860 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:55.189828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc" event={"ID":"eef8e44b-6cd2-4943-b7f2-0444ebc9452d","Type":"ContainerStarted","Data":"5064d033e9286084f5eacbb74ef298e4cded486eb3df961bfffe059b2abd5e06"}
Sep 18 22:22:57.314400 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:57.314346    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 22:22:57.314686 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:57.314435    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 22:22:58.207933 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:58.207895    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc" event={"ID":"eef8e44b-6cd2-4943-b7f2-0444ebc9452d","Type":"ContainerStarted","Data":"fbec611e19df1bd8a071dcdf44a5ff0ea020b795f9c018e79aef31fe2f810877"}
Sep 18 22:22:58.208115 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:58.208097    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:22:58.209885 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:58.209859    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-z44wc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 22:22:58.210011 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:58.209916    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc" podUID="eef8e44b-6cd2-4943-b7f2-0444ebc9452d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 22:22:58.245557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:58.245502    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc" podStartSLOduration=2.090402102 podStartE2EDuration="4.24548592s" totalImagesPullingTime="2.155083818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:22:58.242552659 +0000 UTC m=+362.603069518" watchObservedRunningTime="2026-09-18 22:22:58.24548592 +0000 UTC m=+362.606002776"
Sep 18 22:22:59.216747 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:22:59.216715    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z44wc"
Sep 18 22:23:02.176466 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:02.176438    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-89gq7"
Sep 18 22:23:04.189100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:04.189071    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-m6sss"
Sep 18 22:23:53.383482 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.383449    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"]
Sep 18 22:23:53.386901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.386884    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.389757 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.389736    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 22:23:53.390544 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.390524    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vzhmn\""
Sep 18 22:23:53.390544 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.390539    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 22:23:53.397849 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.397825    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"]
Sep 18 22:23:53.485865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.485832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv8zs\" (UniqueName: \"kubernetes.io/projected/23afa2c3-573d-40ec-b71e-3b6f5682b769-kube-api-access-xv8zs\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.486027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.485885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.486027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.485967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.503192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.503161    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"]
Sep 18 22:23:53.506628 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.506612    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.520169 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.520146    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"]
Sep 18 22:23:53.586927 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.586895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmz48\" (UniqueName: \"kubernetes.io/projected/47e93332-01d2-42ae-8a5a-4b8f2f41d811-kube-api-access-jmz48\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.586927 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.586930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv8zs\" (UniqueName: \"kubernetes.io/projected/23afa2c3-573d-40ec-b71e-3b6f5682b769-kube-api-access-xv8zs\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.587167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.586949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.587167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.587035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.587167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.587093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.587167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.587131    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.587332 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.587315    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.587371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.587354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.593306 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.593283    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"]
Sep 18 22:23:53.596743 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.596723    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.597251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.597234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv8zs\" (UniqueName: \"kubernetes.io/projected/23afa2c3-573d-40ec-b71e-3b6f5682b769-kube-api-access-xv8zs\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.607938 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.607918    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"]
Sep 18 22:23:53.688079 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.687988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmz48\" (UniqueName: \"kubernetes.io/projected/47e93332-01d2-42ae-8a5a-4b8f2f41d811-kube-api-access-jmz48\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.688079 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688025    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.688079 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688048    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4msr5\" (UniqueName: \"kubernetes.io/projected/ac744594-18fe-4f6b-993c-ffb00298f4a9-kube-api-access-4msr5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.688396 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.688396 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.688396 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688191    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.688533 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688480    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.688685 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.688664    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.696667 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.696638    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:53.698614 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.698595    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"]
Sep 18 22:23:53.701874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.701856    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.703921 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.703898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmz48\" (UniqueName: \"kubernetes.io/projected/47e93332-01d2-42ae-8a5a-4b8f2f41d811-kube-api-access-jmz48\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.715224 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.715192    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"]
Sep 18 22:23:53.789051 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msr5\" (UniqueName: \"kubernetes.io/projected/ac744594-18fe-4f6b-993c-ffb00298f4a9-kube-api-access-4msr5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.789220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.789220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789117    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.789220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789170    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.789401 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789333    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgq9z\" (UniqueName: \"kubernetes.io/projected/a43f9aa9-790c-42e9-9df9-cab38e5499ca-kube-api-access-xgq9z\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.789456 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.789624 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789600    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.789688 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.789604    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.798425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.798403    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4msr5\" (UniqueName: \"kubernetes.io/projected/ac744594-18fe-4f6b-993c-ffb00298f4a9-kube-api-access-4msr5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.816231 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.816209    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:53.890040 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.890009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.890168 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.890059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgq9z\" (UniqueName: \"kubernetes.io/projected/a43f9aa9-790c-42e9-9df9-cab38e5499ca-kube-api-access-xgq9z\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.890168 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.890085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.890533 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.890504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.890533 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.890510    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.898805 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.898784    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgq9z\" (UniqueName: \"kubernetes.io/projected/a43f9aa9-790c-42e9-9df9-cab38e5499ca-kube-api-access-xgq9z\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:53.913571 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.913551    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:53.972268 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:53.971803    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"]
Sep 18 22:23:53.975886 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:23:53.975855    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23afa2c3_573d_40ec_b71e_3b6f5682b769.slice/crio-de5563e2f01db9d6e22e80d24745b888c6b34f3a1605efc39e795c76835d67bf WatchSource:0}: Error finding container de5563e2f01db9d6e22e80d24745b888c6b34f3a1605efc39e795c76835d67bf: Status 404 returned error can't find the container with id de5563e2f01db9d6e22e80d24745b888c6b34f3a1605efc39e795c76835d67bf
Sep 18 22:23:54.027135 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.027114    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:54.090753 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.090673    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"]
Sep 18 22:23:54.125205 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:23:54.125177    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e93332_01d2_42ae_8a5a_4b8f2f41d811.slice/crio-55cbbb5b723a9c9ef473f95fa804e4d019864155ec76a0ed21f3087402cd0c17 WatchSource:0}: Error finding container 55cbbb5b723a9c9ef473f95fa804e4d019864155ec76a0ed21f3087402cd0c17: Status 404 returned error can't find the container with id 55cbbb5b723a9c9ef473f95fa804e4d019864155ec76a0ed21f3087402cd0c17
Sep 18 22:23:54.188930 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.188908    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"]
Sep 18 22:23:54.190930 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:23:54.190902    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac744594_18fe_4f6b_993c_ffb00298f4a9.slice/crio-cf4b7d7a5b3ff61b372cd78cd393115465283c261dad1ed524c9ced02219274d WatchSource:0}: Error finding container cf4b7d7a5b3ff61b372cd78cd393115465283c261dad1ed524c9ced02219274d: Status 404 returned error can't find the container with id cf4b7d7a5b3ff61b372cd78cd393115465283c261dad1ed524c9ced02219274d
Sep 18 22:23:54.301336 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.301313    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"]
Sep 18 22:23:54.302671 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:23:54.302645    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43f9aa9_790c_42e9_9df9_cab38e5499ca.slice/crio-657002bf02e89036156763aff7869fbf3f260096eb133c2214f207c22f3c5dee WatchSource:0}: Error finding container 657002bf02e89036156763aff7869fbf3f260096eb133c2214f207c22f3c5dee: Status 404 returned error can't find the container with id 657002bf02e89036156763aff7869fbf3f260096eb133c2214f207c22f3c5dee
Sep 18 22:23:54.472323 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.472287    2583 generic.go:356] "Generic (PLEG): container finished" podID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerID="703ddb25b4912426488bb9b21229b59bed18630ce58b382db4c786e24ea210fa" exitCode=0
Sep 18 22:23:54.472719 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.472405    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb" event={"ID":"47e93332-01d2-42ae-8a5a-4b8f2f41d811","Type":"ContainerDied","Data":"703ddb25b4912426488bb9b21229b59bed18630ce58b382db4c786e24ea210fa"}
Sep 18 22:23:54.472719 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.472465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb" event={"ID":"47e93332-01d2-42ae-8a5a-4b8f2f41d811","Type":"ContainerStarted","Data":"55cbbb5b723a9c9ef473f95fa804e4d019864155ec76a0ed21f3087402cd0c17"}
Sep 18 22:23:54.474189 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.474162    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4" event={"ID":"a43f9aa9-790c-42e9-9df9-cab38e5499ca","Type":"ContainerStarted","Data":"04087ec69751878471ebacdcf36862e48a68046fb88656ed27ba3614ad6356ff"}
Sep 18 22:23:54.474309 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.474211    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4" event={"ID":"a43f9aa9-790c-42e9-9df9-cab38e5499ca","Type":"ContainerStarted","Data":"657002bf02e89036156763aff7869fbf3f260096eb133c2214f207c22f3c5dee"}
Sep 18 22:23:54.475680 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.475662    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerID="f5f71c15069a69270fefa25f55e4ed22c0ddcff4cee2e7872014ad15b74502ba" exitCode=0
Sep 18 22:23:54.475796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.475741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk" event={"ID":"ac744594-18fe-4f6b-993c-ffb00298f4a9","Type":"ContainerDied","Data":"f5f71c15069a69270fefa25f55e4ed22c0ddcff4cee2e7872014ad15b74502ba"}
Sep 18 22:23:54.475796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.475777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk" event={"ID":"ac744594-18fe-4f6b-993c-ffb00298f4a9","Type":"ContainerStarted","Data":"cf4b7d7a5b3ff61b372cd78cd393115465283c261dad1ed524c9ced02219274d"}
Sep 18 22:23:54.477473 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.477452    2583 generic.go:356] "Generic (PLEG): container finished" podID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerID="50cb3a76479c0206c98ce623ae497da5254cfb358cb10220d215de982589fd73" exitCode=0
Sep 18 22:23:54.477545 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.477494    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs" event={"ID":"23afa2c3-573d-40ec-b71e-3b6f5682b769","Type":"ContainerDied","Data":"50cb3a76479c0206c98ce623ae497da5254cfb358cb10220d215de982589fd73"}
Sep 18 22:23:54.477545 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:54.477525    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs" event={"ID":"23afa2c3-573d-40ec-b71e-3b6f5682b769","Type":"ContainerStarted","Data":"de5563e2f01db9d6e22e80d24745b888c6b34f3a1605efc39e795c76835d67bf"}
Sep 18 22:23:55.484846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:55.484812    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb" event={"ID":"47e93332-01d2-42ae-8a5a-4b8f2f41d811","Type":"ContainerStarted","Data":"fc71a0ef9332a670cead7aa30c2a1d85c892e7471ac52ebb44ece7949e3a5ff0"}
Sep 18 22:23:55.486379 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:55.486351    2583 generic.go:356] "Generic (PLEG): container finished" podID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerID="04087ec69751878471ebacdcf36862e48a68046fb88656ed27ba3614ad6356ff" exitCode=0
Sep 18 22:23:55.486505 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:55.486424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4" event={"ID":"a43f9aa9-790c-42e9-9df9-cab38e5499ca","Type":"ContainerDied","Data":"04087ec69751878471ebacdcf36862e48a68046fb88656ed27ba3614ad6356ff"}
Sep 18 22:23:55.488218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:55.488199    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk" event={"ID":"ac744594-18fe-4f6b-993c-ffb00298f4a9","Type":"ContainerStarted","Data":"9f8e3788063c9374cb8ee59b76ba559bbec1ff445c669703098d4cf37860eea7"}
Sep 18 22:23:56.494976 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.494941    2583 generic.go:356] "Generic (PLEG): container finished" podID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerID="295cc709ecde3c9e40122319cc0e9aacecba44d9f5888ef9824bbc8e0ec07866" exitCode=0
Sep 18 22:23:56.495433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.495029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4" event={"ID":"a43f9aa9-790c-42e9-9df9-cab38e5499ca","Type":"ContainerDied","Data":"295cc709ecde3c9e40122319cc0e9aacecba44d9f5888ef9824bbc8e0ec07866"}
Sep 18 22:23:56.496999 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.496978    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerID="9f8e3788063c9374cb8ee59b76ba559bbec1ff445c669703098d4cf37860eea7" exitCode=0
Sep 18 22:23:56.497099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.497060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk" event={"ID":"ac744594-18fe-4f6b-993c-ffb00298f4a9","Type":"ContainerDied","Data":"9f8e3788063c9374cb8ee59b76ba559bbec1ff445c669703098d4cf37860eea7"}
Sep 18 22:23:56.499150 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.499129    2583 generic.go:356] "Generic (PLEG): container finished" podID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerID="c305b7e803bac8e579f6ce37f9e81c5efa26d8ff27988aaed093ab491af59695" exitCode=0
Sep 18 22:23:56.499251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.499205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs" event={"ID":"23afa2c3-573d-40ec-b71e-3b6f5682b769","Type":"ContainerDied","Data":"c305b7e803bac8e579f6ce37f9e81c5efa26d8ff27988aaed093ab491af59695"}
Sep 18 22:23:56.502156 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.501608    2583 generic.go:356] "Generic (PLEG): container finished" podID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerID="fc71a0ef9332a670cead7aa30c2a1d85c892e7471ac52ebb44ece7949e3a5ff0" exitCode=0
Sep 18 22:23:56.502156 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:56.501656    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb" event={"ID":"47e93332-01d2-42ae-8a5a-4b8f2f41d811","Type":"ContainerDied","Data":"fc71a0ef9332a670cead7aa30c2a1d85c892e7471ac52ebb44ece7949e3a5ff0"}
Sep 18 22:23:57.508732 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.508694    2583 generic.go:356] "Generic (PLEG): container finished" podID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerID="2cd60f0b46475edc147d8280a1c88846c9668d84ac674cc1cfc624dc771f5837" exitCode=0
Sep 18 22:23:57.509105 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.508778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs" event={"ID":"23afa2c3-573d-40ec-b71e-3b6f5682b769","Type":"ContainerDied","Data":"2cd60f0b46475edc147d8280a1c88846c9668d84ac674cc1cfc624dc771f5837"}
Sep 18 22:23:57.510745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.510722    2583 generic.go:356] "Generic (PLEG): container finished" podID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerID="0b0f24bf74fe71259509423408a3c6d80b8f39ed7a433ece0e31fa83732d10ee" exitCode=0
Sep 18 22:23:57.510853 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.510803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb" event={"ID":"47e93332-01d2-42ae-8a5a-4b8f2f41d811","Type":"ContainerDied","Data":"0b0f24bf74fe71259509423408a3c6d80b8f39ed7a433ece0e31fa83732d10ee"}
Sep 18 22:23:57.512748 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.512728    2583 generic.go:356] "Generic (PLEG): container finished" podID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerID="e2cdfb1393aeb0177811c9f9ea831984d2d7cb2309fdc3957294c8aa924a4a65" exitCode=0
Sep 18 22:23:57.512845 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.512787    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4" event={"ID":"a43f9aa9-790c-42e9-9df9-cab38e5499ca","Type":"ContainerDied","Data":"e2cdfb1393aeb0177811c9f9ea831984d2d7cb2309fdc3957294c8aa924a4a65"}
Sep 18 22:23:57.514646 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.514619    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerID="7d830bdb0b9cf2818274b284e538f960daaefb58fca47cfa4f68d527ef61629d" exitCode=0
Sep 18 22:23:57.514717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:57.514663    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk" event={"ID":"ac744594-18fe-4f6b-993c-ffb00298f4a9","Type":"ContainerDied","Data":"7d830bdb0b9cf2818274b284e538f960daaefb58fca47cfa4f68d527ef61629d"}
Sep 18 22:23:58.699317 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.699134    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:58.701486 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.701462    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:58.724176 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.724152    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:58.728256 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.728236    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:58.843149 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843061    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmz48\" (UniqueName: \"kubernetes.io/projected/47e93332-01d2-42ae-8a5a-4b8f2f41d811-kube-api-access-jmz48\") pod \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") "
Sep 18 22:23:58.843149 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843102    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-bundle\") pod \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") "
Sep 18 22:23:58.843149 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843142    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-util\") pod \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\" (UID: \"47e93332-01d2-42ae-8a5a-4b8f2f41d811\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843163    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-util\") pod \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843194    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-util\") pod \"ac744594-18fe-4f6b-993c-ffb00298f4a9\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843222    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgq9z\" (UniqueName: \"kubernetes.io/projected/a43f9aa9-790c-42e9-9df9-cab38e5499ca-kube-api-access-xgq9z\") pod \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843271    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4msr5\" (UniqueName: \"kubernetes.io/projected/ac744594-18fe-4f6b-993c-ffb00298f4a9-kube-api-access-4msr5\") pod \"ac744594-18fe-4f6b-993c-ffb00298f4a9\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843292    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-bundle\") pod \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\" (UID: \"a43f9aa9-790c-42e9-9df9-cab38e5499ca\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843327    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-util\") pod \"23afa2c3-573d-40ec-b71e-3b6f5682b769\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843363    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-bundle\") pod \"23afa2c3-573d-40ec-b71e-3b6f5682b769\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") "
Sep 18 22:23:58.843452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843413    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv8zs\" (UniqueName: \"kubernetes.io/projected/23afa2c3-573d-40ec-b71e-3b6f5682b769-kube-api-access-xv8zs\") pod \"23afa2c3-573d-40ec-b71e-3b6f5682b769\" (UID: \"23afa2c3-573d-40ec-b71e-3b6f5682b769\") "
Sep 18 22:23:58.843814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843459    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-bundle\") pod \"ac744594-18fe-4f6b-993c-ffb00298f4a9\" (UID: \"ac744594-18fe-4f6b-993c-ffb00298f4a9\") "
Sep 18 22:23:58.843876 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.843838    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-bundle" (OuterVolumeSpecName: "bundle") pod "47e93332-01d2-42ae-8a5a-4b8f2f41d811" (UID: "47e93332-01d2-42ae-8a5a-4b8f2f41d811"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.844236 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.844190    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-bundle" (OuterVolumeSpecName: "bundle") pod "ac744594-18fe-4f6b-993c-ffb00298f4a9" (UID: "ac744594-18fe-4f6b-993c-ffb00298f4a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.844674 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.844643    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-bundle" (OuterVolumeSpecName: "bundle") pod "23afa2c3-573d-40ec-b71e-3b6f5682b769" (UID: "23afa2c3-573d-40ec-b71e-3b6f5682b769"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.845428 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.845399    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-bundle" (OuterVolumeSpecName: "bundle") pod "a43f9aa9-790c-42e9-9df9-cab38e5499ca" (UID: "a43f9aa9-790c-42e9-9df9-cab38e5499ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.845870 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.845842    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e93332-01d2-42ae-8a5a-4b8f2f41d811-kube-api-access-jmz48" (OuterVolumeSpecName: "kube-api-access-jmz48") pod "47e93332-01d2-42ae-8a5a-4b8f2f41d811" (UID: "47e93332-01d2-42ae-8a5a-4b8f2f41d811"). InnerVolumeSpecName "kube-api-access-jmz48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:23:58.846488 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.846458    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23afa2c3-573d-40ec-b71e-3b6f5682b769-kube-api-access-xv8zs" (OuterVolumeSpecName: "kube-api-access-xv8zs") pod "23afa2c3-573d-40ec-b71e-3b6f5682b769" (UID: "23afa2c3-573d-40ec-b71e-3b6f5682b769"). InnerVolumeSpecName "kube-api-access-xv8zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:23:58.847313 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.847289    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43f9aa9-790c-42e9-9df9-cab38e5499ca-kube-api-access-xgq9z" (OuterVolumeSpecName: "kube-api-access-xgq9z") pod "a43f9aa9-790c-42e9-9df9-cab38e5499ca" (UID: "a43f9aa9-790c-42e9-9df9-cab38e5499ca"). InnerVolumeSpecName "kube-api-access-xgq9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:23:58.847547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.847511    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac744594-18fe-4f6b-993c-ffb00298f4a9-kube-api-access-4msr5" (OuterVolumeSpecName: "kube-api-access-4msr5") pod "ac744594-18fe-4f6b-993c-ffb00298f4a9" (UID: "ac744594-18fe-4f6b-993c-ffb00298f4a9"). InnerVolumeSpecName "kube-api-access-4msr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:23:58.851335 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.851295    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-util" (OuterVolumeSpecName: "util") pod "47e93332-01d2-42ae-8a5a-4b8f2f41d811" (UID: "47e93332-01d2-42ae-8a5a-4b8f2f41d811"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.852003 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.851969    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-util" (OuterVolumeSpecName: "util") pod "a43f9aa9-790c-42e9-9df9-cab38e5499ca" (UID: "a43f9aa9-790c-42e9-9df9-cab38e5499ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.852558 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.852536    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-util" (OuterVolumeSpecName: "util") pod "23afa2c3-573d-40ec-b71e-3b6f5682b769" (UID: "23afa2c3-573d-40ec-b71e-3b6f5682b769"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.852607 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.852594    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-util" (OuterVolumeSpecName: "util") pod "ac744594-18fe-4f6b-993c-ffb00298f4a9" (UID: "ac744594-18fe-4f6b-993c-ffb00298f4a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:23:58.944679 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944645    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmz48\" (UniqueName: \"kubernetes.io/projected/47e93332-01d2-42ae-8a5a-4b8f2f41d811-kube-api-access-jmz48\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944679 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944672    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944679 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944686    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47e93332-01d2-42ae-8a5a-4b8f2f41d811-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944697    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944708    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944719    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgq9z\" (UniqueName: \"kubernetes.io/projected/a43f9aa9-790c-42e9-9df9-cab38e5499ca-kube-api-access-xgq9z\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944730    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4msr5\" (UniqueName: \"kubernetes.io/projected/ac744594-18fe-4f6b-993c-ffb00298f4a9-kube-api-access-4msr5\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944743    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a43f9aa9-790c-42e9-9df9-cab38e5499ca-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944753    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944763    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23afa2c3-573d-40ec-b71e-3b6f5682b769-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944773    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv8zs\" (UniqueName: \"kubernetes.io/projected/23afa2c3-573d-40ec-b71e-3b6f5682b769-kube-api-access-xv8zs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:58.944908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:58.944783    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac744594-18fe-4f6b-993c-ffb00298f4a9-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:23:59.526717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.526685    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs"
Sep 18 22:23:59.526904 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.526679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8brs" event={"ID":"23afa2c3-573d-40ec-b71e-3b6f5682b769","Type":"ContainerDied","Data":"de5563e2f01db9d6e22e80d24745b888c6b34f3a1605efc39e795c76835d67bf"}
Sep 18 22:23:59.526904 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.526850    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de5563e2f01db9d6e22e80d24745b888c6b34f3a1605efc39e795c76835d67bf"
Sep 18 22:23:59.528791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.528772    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb"
Sep 18 22:23:59.528919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.528806    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gh2lb" event={"ID":"47e93332-01d2-42ae-8a5a-4b8f2f41d811","Type":"ContainerDied","Data":"55cbbb5b723a9c9ef473f95fa804e4d019864155ec76a0ed21f3087402cd0c17"}
Sep 18 22:23:59.528919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.528855    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55cbbb5b723a9c9ef473f95fa804e4d019864155ec76a0ed21f3087402cd0c17"
Sep 18 22:23:59.530945 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.530924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4" event={"ID":"a43f9aa9-790c-42e9-9df9-cab38e5499ca","Type":"ContainerDied","Data":"657002bf02e89036156763aff7869fbf3f260096eb133c2214f207c22f3c5dee"}
Sep 18 22:23:59.530945 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.530939    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cst5z4"
Sep 18 22:23:59.531097 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.530959    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="657002bf02e89036156763aff7869fbf3f260096eb133c2214f207c22f3c5dee"
Sep 18 22:23:59.533119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.533100    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk" event={"ID":"ac744594-18fe-4f6b-993c-ffb00298f4a9","Type":"ContainerDied","Data":"cf4b7d7a5b3ff61b372cd78cd393115465283c261dad1ed524c9ced02219274d"}
Sep 18 22:23:59.533119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.533115    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66q8nk"
Sep 18 22:23:59.533286 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:23:59.533130    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf4b7d7a5b3ff61b372cd78cd393115465283c261dad1ed524c9ced02219274d"
Sep 18 22:25:41.235244 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235162    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8567457bbb-5ghhv"]
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235477    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235491    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235506    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235512    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235521    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235526    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235536    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235541    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235547    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235552    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235559    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235564    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235573    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235578    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235584    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235590    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235596    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235601    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235610    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235614    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235622    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235627    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="util"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235632    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235637    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="pull"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235688    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e93332-01d2-42ae-8a5a-4b8f2f41d811" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235696    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43f9aa9-790c-42e9-9df9-cab38e5499ca" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235703    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac744594-18fe-4f6b-993c-ffb00298f4a9" containerName="extract"
Sep 18 22:25:41.235696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.235710    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="23afa2c3-573d-40ec-b71e-3b6f5682b769" containerName="extract"
Sep 18 22:25:41.238602 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.238586    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.244180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.244147    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 22:25:41.244313 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.244294    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 22:25:41.245596 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.245185    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lqtmn\""
Sep 18 22:25:41.245596 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.245301    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 22:25:41.252896 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.252872    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8567457bbb-5ghhv"]
Sep 18 22:25:41.278864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.278837    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b49kl\" (UniqueName: \"kubernetes.io/projected/d6fe8c04-97d5-4a24-9588-5dde268a6a79-kube-api-access-b49kl\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.278996 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.278876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-webhook-cert\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.278996 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.278941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-metrics-certs\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.380249 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.380213    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b49kl\" (UniqueName: \"kubernetes.io/projected/d6fe8c04-97d5-4a24-9588-5dde268a6a79-kube-api-access-b49kl\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.380445 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.380270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-webhook-cert\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.380445 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.380323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-metrics-certs\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.382809 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.382787    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-webhook-cert\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.382876 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.382842    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-metrics-certs\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.389145 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.389123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b49kl\" (UniqueName: \"kubernetes.io/projected/d6fe8c04-97d5-4a24-9588-5dde268a6a79-kube-api-access-b49kl\") pod \"maas-controller-8567457bbb-5ghhv\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") " pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.548640 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.548560    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:41.821696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:41.821629    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8567457bbb-5ghhv"]
Sep 18 22:25:41.823860 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:25:41.823834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fe8c04_97d5_4a24_9588_5dde268a6a79.slice/crio-88a3417e3b4bf65f53573d23a1ce8348c93938332ddec6a417c2f23012c16cdf WatchSource:0}: Error finding container 88a3417e3b4bf65f53573d23a1ce8348c93938332ddec6a417c2f23012c16cdf: Status 404 returned error can't find the container with id 88a3417e3b4bf65f53573d23a1ce8348c93938332ddec6a417c2f23012c16cdf
Sep 18 22:25:42.037725 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:42.037694    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8567457bbb-5ghhv" event={"ID":"d6fe8c04-97d5-4a24-9588-5dde268a6a79","Type":"ContainerStarted","Data":"88a3417e3b4bf65f53573d23a1ce8348c93938332ddec6a417c2f23012c16cdf"}
Sep 18 22:25:45.055815 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:45.055775    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8567457bbb-5ghhv" event={"ID":"d6fe8c04-97d5-4a24-9588-5dde268a6a79","Type":"ContainerStarted","Data":"89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4"}
Sep 18 22:25:45.056201 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:45.055901    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:25:45.073078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:45.073022    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8567457bbb-5ghhv" podStartSLOduration=1.8213843939999999 podStartE2EDuration="4.073003673s" totalImagesPullingTime="2.251619279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:25:45.072514232 +0000 UTC m=+529.433031089" watchObservedRunningTime="2026-09-18 22:25:45.073003673 +0000 UTC m=+529.433520532"
Sep 18 22:25:56.066629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:25:56.066600    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:26:15.902428 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.902380    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"]
Sep 18 22:26:15.905717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.905696    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:15.910119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.910098    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9km2v\""
Sep 18 22:26:15.910231 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.910107    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 22:26:15.920626 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.920604    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-twfmq"]
Sep 18 22:26:15.923853 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.923831    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:15.925081 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.925055    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"]
Sep 18 22:26:15.945706 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:15.945680    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-twfmq"]
Sep 18 22:26:16.065158 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.065121    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904be353-e4a5-48fe-9054-397522c3dd4f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-qsknm\" (UID: \"904be353-e4a5-48fe-9054-397522c3dd4f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.065158 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.065174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdnf9\" (UniqueName: \"kubernetes.io/projected/904be353-e4a5-48fe-9054-397522c3dd4f-kube-api-access-qdnf9\") pod \"payload-pre-processing-59fbc47bf8-qsknm\" (UID: \"904be353-e4a5-48fe-9054-397522c3dd4f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.065374 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.065253    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80653214-eddf-4e73-882c-4348f6613d7e-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-twfmq\" (UID: \"80653214-eddf-4e73-882c-4348f6613d7e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.065374 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.065286    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdmpn\" (UniqueName: \"kubernetes.io/projected/80653214-eddf-4e73-882c-4348f6613d7e-kube-api-access-xdmpn\") pod \"payload-processing-5b54b7fbb7-twfmq\" (UID: \"80653214-eddf-4e73-882c-4348f6613d7e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.166371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.166282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80653214-eddf-4e73-882c-4348f6613d7e-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-twfmq\" (UID: \"80653214-eddf-4e73-882c-4348f6613d7e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.166371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.166320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdmpn\" (UniqueName: \"kubernetes.io/projected/80653214-eddf-4e73-882c-4348f6613d7e-kube-api-access-xdmpn\") pod \"payload-processing-5b54b7fbb7-twfmq\" (UID: \"80653214-eddf-4e73-882c-4348f6613d7e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.166615 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.166373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904be353-e4a5-48fe-9054-397522c3dd4f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-qsknm\" (UID: \"904be353-e4a5-48fe-9054-397522c3dd4f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.166615 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.166429    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdnf9\" (UniqueName: \"kubernetes.io/projected/904be353-e4a5-48fe-9054-397522c3dd4f-kube-api-access-qdnf9\") pod \"payload-pre-processing-59fbc47bf8-qsknm\" (UID: \"904be353-e4a5-48fe-9054-397522c3dd4f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.167054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.167032    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80653214-eddf-4e73-882c-4348f6613d7e-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-twfmq\" (UID: \"80653214-eddf-4e73-882c-4348f6613d7e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.167207 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.167185    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904be353-e4a5-48fe-9054-397522c3dd4f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-qsknm\" (UID: \"904be353-e4a5-48fe-9054-397522c3dd4f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.180454 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.180429    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdnf9\" (UniqueName: \"kubernetes.io/projected/904be353-e4a5-48fe-9054-397522c3dd4f-kube-api-access-qdnf9\") pod \"payload-pre-processing-59fbc47bf8-qsknm\" (UID: \"904be353-e4a5-48fe-9054-397522c3dd4f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.180625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.180604    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdmpn\" (UniqueName: \"kubernetes.io/projected/80653214-eddf-4e73-882c-4348f6613d7e-kube-api-access-xdmpn\") pod \"payload-processing-5b54b7fbb7-twfmq\" (UID: \"80653214-eddf-4e73-882c-4348f6613d7e\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.215758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.215732    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:16.235782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.235756    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:16.495178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.495152    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"]
Sep 18 22:26:16.496600 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:26:16.496574    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904be353_e4a5_48fe_9054_397522c3dd4f.slice/crio-2b6f51eb072145b8faebeede612a07be5bddaf335ed4dfcbe902c09259a2747f WatchSource:0}: Error finding container 2b6f51eb072145b8faebeede612a07be5bddaf335ed4dfcbe902c09259a2747f: Status 404 returned error can't find the container with id 2b6f51eb072145b8faebeede612a07be5bddaf335ed4dfcbe902c09259a2747f
Sep 18 22:26:16.525030 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:16.525006    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-twfmq"]
Sep 18 22:26:16.526506 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:26:16.526479    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80653214_eddf_4e73_882c_4348f6613d7e.slice/crio-7658446bcd510ada857af87a6221223160b119d3509d42f73c64846ffe66201e WatchSource:0}: Error finding container 7658446bcd510ada857af87a6221223160b119d3509d42f73c64846ffe66201e: Status 404 returned error can't find the container with id 7658446bcd510ada857af87a6221223160b119d3509d42f73c64846ffe66201e
Sep 18 22:26:17.217033 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:17.216493    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm" event={"ID":"904be353-e4a5-48fe-9054-397522c3dd4f","Type":"ContainerStarted","Data":"2b6f51eb072145b8faebeede612a07be5bddaf335ed4dfcbe902c09259a2747f"}
Sep 18 22:26:17.218146 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:17.218119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq" event={"ID":"80653214-eddf-4e73-882c-4348f6613d7e","Type":"ContainerStarted","Data":"7658446bcd510ada857af87a6221223160b119d3509d42f73c64846ffe66201e"}
Sep 18 22:26:18.224928 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:18.224888    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq" event={"ID":"80653214-eddf-4e73-882c-4348f6613d7e","Type":"ContainerStarted","Data":"5661b352b7b995320d8864db4529ea654bc2ad41aa4b298b28ea8b8ada8539c9"}
Sep 18 22:26:18.225419 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:18.224976    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:18.226598 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:18.226565    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm" event={"ID":"904be353-e4a5-48fe-9054-397522c3dd4f","Type":"ContainerStarted","Data":"14fbbf1368bfe0270ae83ea16a8a4053e925afdbbb9e20be8302aa4fe1375166"}
Sep 18 22:26:18.226749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:18.226620    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:18.244403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:18.244336    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq" podStartSLOduration=2.124951572 podStartE2EDuration="3.244318727s" totalImagesPullingTime="1.119367155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:26:18.243109226 +0000 UTC m=+562.603626078" watchObservedRunningTime="2026-09-18 22:26:18.244318727 +0000 UTC m=+562.604835585"
Sep 18 22:26:18.261688 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:18.261636    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm" podStartSLOduration=2.114610466 podStartE2EDuration="3.261616141s" totalImagesPullingTime="1.147005675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:26:18.259570671 +0000 UTC m=+562.620087529" watchObservedRunningTime="2026-09-18 22:26:18.261616141 +0000 UTC m=+562.622132999"
Sep 18 22:26:29.234220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:29.234189    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qsknm"
Sep 18 22:26:29.234632 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:29.234261    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-twfmq"
Sep 18 22:26:57.584813 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:57.584787    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:26:57.585365 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:26:57.584988    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:27:24.526844 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.526749    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"]
Sep 18 22:27:24.530245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.530224    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.533982 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.533958    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 18 22:27:24.533982 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.533984    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 22:27:24.534179 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.534095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s9rh9\""
Sep 18 22:27:24.534238 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.534184    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 22:27:24.541274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.541249    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"]
Sep 18 22:27:24.640081 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.640040    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.640273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.640087    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kn5g\" (UniqueName: \"kubernetes.io/projected/37224245-8fbc-419b-b588-1b369fc24f1f-kube-api-access-4kn5g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.640273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.640140    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.640273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.640187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.640273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.640232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37224245-8fbc-419b-b588-1b369fc24f1f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.640273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.640254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.741567 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.741758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741576    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kn5g\" (UniqueName: \"kubernetes.io/projected/37224245-8fbc-419b-b588-1b369fc24f1f-kube-api-access-4kn5g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.741758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.741758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.741758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37224245-8fbc-419b-b588-1b369fc24f1f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.741758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.742012 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.741967    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.742012 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.742002    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.742107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.742086    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.744115 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.744081    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37224245-8fbc-419b-b588-1b369fc24f1f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.744296 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.744276    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37224245-8fbc-419b-b588-1b369fc24f1f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.750572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.750550    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kn5g\" (UniqueName: \"kubernetes.io/projected/37224245-8fbc-419b-b588-1b369fc24f1f-kube-api-access-4kn5g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jnsqt\" (UID: \"37224245-8fbc-419b-b588-1b369fc24f1f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:24.843595 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:24.843519    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:25.118432 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:25.118381    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"]
Sep 18 22:27:25.120297 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:27:25.120270    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37224245_8fbc_419b_b588_1b369fc24f1f.slice/crio-1672eb76da68895337fe90f2e6125a4ea0bbe23ea999d9afba0158be6f5839a2 WatchSource:0}: Error finding container 1672eb76da68895337fe90f2e6125a4ea0bbe23ea999d9afba0158be6f5839a2: Status 404 returned error can't find the container with id 1672eb76da68895337fe90f2e6125a4ea0bbe23ea999d9afba0158be6f5839a2
Sep 18 22:27:25.121585 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:25.121570    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:27:25.567335 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:25.567295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt" event={"ID":"37224245-8fbc-419b-b588-1b369fc24f1f","Type":"ContainerStarted","Data":"1672eb76da68895337fe90f2e6125a4ea0bbe23ea999d9afba0158be6f5839a2"}
Sep 18 22:27:26.574790 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:26.574753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt" event={"ID":"37224245-8fbc-419b-b588-1b369fc24f1f","Type":"ContainerStarted","Data":"7e0e78327e37dad7259ab5588d02a89165032b17b9306bdd615073da6d63e915"}
Sep 18 22:27:26.624708 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:26.624657    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt" podStartSLOduration=1.669358002 podStartE2EDuration="2.624638833s" totalImagesPullingTime="955.280831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:27:26.623539743 +0000 UTC m=+630.984056601" watchObservedRunningTime="2026-09-18 22:27:26.624638833 +0000 UTC m=+630.985155693"
Sep 18 22:27:34.843876 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:34.843792    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:34.844296 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:34.844089    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:34.856793 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:34.856768    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:35.631305 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:35.631277    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jnsqt"
Sep 18 22:27:44.644287 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.644253    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"]
Sep 18 22:27:44.648254 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.648233    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.650935 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.650913    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 22:27:44.659332 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.659308    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"]
Sep 18 22:27:44.706119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.706085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.706288 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.706125    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.706288 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.706204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwwn4\" (UniqueName: \"kubernetes.io/projected/041109bc-773d-4cda-a03f-679eaec8b8f2-kube-api-access-mwwn4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.706288 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.706239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/041109bc-773d-4cda-a03f-679eaec8b8f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.706476 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.706294    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.706476 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.706322    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807040 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwwn4\" (UniqueName: \"kubernetes.io/projected/041109bc-773d-4cda-a03f-679eaec8b8f2-kube-api-access-mwwn4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807040 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807043    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/041109bc-773d-4cda-a03f-679eaec8b8f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807291 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807291 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807291 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807291 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807284    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807638 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807685 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807627    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.807726 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.807712    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.809328 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.809304    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/041109bc-773d-4cda-a03f-679eaec8b8f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.809489 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.809466    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/041109bc-773d-4cda-a03f-679eaec8b8f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.822731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.822705    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwwn4\" (UniqueName: \"kubernetes.io/projected/041109bc-773d-4cda-a03f-679eaec8b8f2-kube-api-access-mwwn4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xsphf\" (UID: \"041109bc-773d-4cda-a03f-679eaec8b8f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:44.959236 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:44.959154    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:45.232551 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:45.232525    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"]
Sep 18 22:27:45.234120 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:27:45.234095    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041109bc_773d_4cda_a03f_679eaec8b8f2.slice/crio-5177c7cc14545b5f6134e1cf4967fb6e9e00d00e22794f0064036c2cb24fbf21 WatchSource:0}: Error finding container 5177c7cc14545b5f6134e1cf4967fb6e9e00d00e22794f0064036c2cb24fbf21: Status 404 returned error can't find the container with id 5177c7cc14545b5f6134e1cf4967fb6e9e00d00e22794f0064036c2cb24fbf21
Sep 18 22:27:45.671849 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:45.671816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf" event={"ID":"041109bc-773d-4cda-a03f-679eaec8b8f2","Type":"ContainerStarted","Data":"e4172416b83db9c7eea4b5647de9cc3ae7cd7dc11cdd6e4089e87c827b5c5fd7"}
Sep 18 22:27:45.672252 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:45.671864    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf" event={"ID":"041109bc-773d-4cda-a03f-679eaec8b8f2","Type":"ContainerStarted","Data":"5177c7cc14545b5f6134e1cf4967fb6e9e00d00e22794f0064036c2cb24fbf21"}
Sep 18 22:27:45.690256 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:45.690210    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf" podStartSLOduration=1.5035864110000001 podStartE2EDuration="1.69019353s" totalImagesPullingTime="186.607119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:27:45.689270209 +0000 UTC m=+650.049787092" watchObservedRunningTime="2026-09-18 22:27:45.69019353 +0000 UTC m=+650.050710387"
Sep 18 22:27:54.960079 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:54.960046    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:54.960553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:54.960117    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:54.973222 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:54.973196    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:27:55.736600 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:27:55.736575    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xsphf"
Sep 18 22:29:11.532854 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.532734    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"]
Sep 18 22:29:11.536512 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.536491    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.539604 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.539584    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 22:29:11.540749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.540729    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 22:29:11.540852 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.540727    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-eac715-serving-cert\""
Sep 18 22:29:11.541185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.541158    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7xxrx\""
Sep 18 22:29:11.541285 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.541189    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 22:29:11.549867 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.549848    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"]
Sep 18 22:29:11.567677 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.567651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-openshift-service-ca\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.567811 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.567748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-metrics-certs\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.567811 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.567781    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.567924 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.567838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrx2b\" (UniqueName: \"kubernetes.io/projected/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-kube-api-access-wrx2b\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.669320 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.669286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-metrics-certs\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.669320 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.669323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.669579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.669346    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrx2b\" (UniqueName: \"kubernetes.io/projected/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-kube-api-access-wrx2b\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.669579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.669451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-openshift-service-ca\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.669579 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:29:11.669477    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-serving-cert: secret "maas-api-e2e-inf-a-eac715-serving-cert" not found
Sep 18 22:29:11.669579 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:29:11.669559    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls podName:bbae3e8f-5a90-43cb-a953-c9adf8a65f53 nodeName:}" failed. No retries permitted until 2026-09-18 22:29:12.169541948 +0000 UTC m=+736.530058787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls") pod "maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" (UID: "bbae3e8f-5a90-43cb-a953-c9adf8a65f53") : secret "maas-api-e2e-inf-a-eac715-serving-cert" not found
Sep 18 22:29:11.670146 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.670121    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-openshift-service-ca\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.671908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.671886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-metrics-certs\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:11.678878 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:11.678850    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrx2b\" (UniqueName: \"kubernetes.io/projected/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-kube-api-access-wrx2b\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:12.173827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.173783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:12.176919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.176892    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls\") pod \"maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:12.300169 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.300137    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"]
Sep 18 22:29:12.303644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.303629    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.306452 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.306367    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-eac715\""
Sep 18 22:29:12.306616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.306598    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-eac715-dockercfg-qm8z9\""
Sep 18 22:29:12.316748 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.316725    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"]
Sep 18 22:29:12.345599 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.345570    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"]
Sep 18 22:29:12.348850 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.348834    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.361011 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.360984    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"]
Sep 18 22:29:12.375276 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.375219    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx9xl\" (UniqueName: \"kubernetes.io/projected/a318e0b6-d78c-4460-9013-866bc2503d81-kube-api-access-jx9xl\") pod \"payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") " pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.375276 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.375257    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a318e0b6-d78c-4460-9013-866bc2503d81-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") " pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.375541 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.375328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ae8c507-8d59-4d72-80b2-a3ea29328130-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.375541 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.375436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr56v\" (UniqueName: \"kubernetes.io/projected/4ae8c507-8d59-4d72-80b2-a3ea29328130-kube-api-access-tr56v\") pod \"payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.448061 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.447990    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:12.476955 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.476918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr56v\" (UniqueName: \"kubernetes.io/projected/4ae8c507-8d59-4d72-80b2-a3ea29328130-kube-api-access-tr56v\") pod \"payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.477107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.477023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx9xl\" (UniqueName: \"kubernetes.io/projected/a318e0b6-d78c-4460-9013-866bc2503d81-kube-api-access-jx9xl\") pod \"payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") " pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.477107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.477069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a318e0b6-d78c-4460-9013-866bc2503d81-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") " pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.477276 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.477145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ae8c507-8d59-4d72-80b2-a3ea29328130-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.477883 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.477858    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a318e0b6-d78c-4460-9013-866bc2503d81-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") " pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.478181 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.478158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ae8c507-8d59-4d72-80b2-a3ea29328130-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.485798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.485767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr56v\" (UniqueName: \"kubernetes.io/projected/4ae8c507-8d59-4d72-80b2-a3ea29328130-kube-api-access-tr56v\") pod \"payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.485911 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.485847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx9xl\" (UniqueName: \"kubernetes.io/projected/a318e0b6-d78c-4460-9013-866bc2503d81-kube-api-access-jx9xl\") pod \"payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") " pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.615040 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.615007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:12.659169 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.659138    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:12.741696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.741650    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"]
Sep 18 22:29:12.742883 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:12.742854    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbae3e8f_5a90_43cb_a953_c9adf8a65f53.slice/crio-ee98d1421fc47218942dfb288d8a670eb557e3fff4bc6fd6f5045b59cc890e08 WatchSource:0}: Error finding container ee98d1421fc47218942dfb288d8a670eb557e3fff4bc6fd6f5045b59cc890e08: Status 404 returned error can't find the container with id ee98d1421fc47218942dfb288d8a670eb557e3fff4bc6fd6f5045b59cc890e08
Sep 18 22:29:12.893367 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.893342    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"]
Sep 18 22:29:12.894894 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:12.894870    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda318e0b6_d78c_4460_9013_866bc2503d81.slice/crio-9160607b01cbb09b48d21056467cc9be4d7aac46f72da03a2afc4ebeffbc4f17 WatchSource:0}: Error finding container 9160607b01cbb09b48d21056467cc9be4d7aac46f72da03a2afc4ebeffbc4f17: Status 404 returned error can't find the container with id 9160607b01cbb09b48d21056467cc9be4d7aac46f72da03a2afc4ebeffbc4f17
Sep 18 22:29:12.952226 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:12.952200    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"]
Sep 18 22:29:12.953033 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:12.953008    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae8c507_8d59_4d72_80b2_a3ea29328130.slice/crio-439c1da1421bcc95176f9db8414d12472eddde290f1d2e43f0bc5c1debb358c3 WatchSource:0}: Error finding container 439c1da1421bcc95176f9db8414d12472eddde290f1d2e43f0bc5c1debb358c3: Status 404 returned error can't find the container with id 439c1da1421bcc95176f9db8414d12472eddde290f1d2e43f0bc5c1debb358c3
Sep 18 22:29:13.124579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.124537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn" event={"ID":"4ae8c507-8d59-4d72-80b2-a3ea29328130","Type":"ContainerStarted","Data":"600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97"}
Sep 18 22:29:13.124765 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.124607    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn" event={"ID":"4ae8c507-8d59-4d72-80b2-a3ea29328130","Type":"ContainerStarted","Data":"439c1da1421bcc95176f9db8414d12472eddde290f1d2e43f0bc5c1debb358c3"}
Sep 18 22:29:13.124765 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.124636    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:13.127029 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.126999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n" event={"ID":"a318e0b6-d78c-4460-9013-866bc2503d81","Type":"ContainerStarted","Data":"e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca"}
Sep 18 22:29:13.127171 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.127055    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n" event={"ID":"a318e0b6-d78c-4460-9013-866bc2503d81","Type":"ContainerStarted","Data":"9160607b01cbb09b48d21056467cc9be4d7aac46f72da03a2afc4ebeffbc4f17"}
Sep 18 22:29:13.127228 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.127170    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:13.128734 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.128710    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" event={"ID":"bbae3e8f-5a90-43cb-a953-c9adf8a65f53","Type":"ContainerStarted","Data":"ee98d1421fc47218942dfb288d8a670eb557e3fff4bc6fd6f5045b59cc890e08"}
Sep 18 22:29:13.151754 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.151691    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn" podStartSLOduration=1.151674338 podStartE2EDuration="1.151674338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:13.150596672 +0000 UTC m=+737.511113530" watchObservedRunningTime="2026-09-18 22:29:13.151674338 +0000 UTC m=+737.512191275"
Sep 18 22:29:13.175936 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.175891    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n" podStartSLOduration=1.175873185 podStartE2EDuration="1.175873185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:13.175018298 +0000 UTC m=+737.535535155" watchObservedRunningTime="2026-09-18 22:29:13.175873185 +0000 UTC m=+737.536390045"
Sep 18 22:29:13.318293 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.318217    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"]
Sep 18 22:29:13.328137 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.328099    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.331557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.331530    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-f9cf33-dockercfg-8v2sk\""
Sep 18 22:29:13.331803 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.331779    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-f9cf33\""
Sep 18 22:29:13.335661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.334887    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"]
Sep 18 22:29:13.384966 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.384932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltnn4\" (UniqueName: \"kubernetes.io/projected/64c838bb-54de-491c-9a76-e7989c79f563-kube-api-access-ltnn4\") pod \"payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.385169 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.384999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c838bb-54de-491c-9a76-e7989c79f563-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.387425 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.387399    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"]
Sep 18 22:29:13.390693 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.390677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.402326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.402305    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"]
Sep 18 22:29:13.486192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.486164    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r88mm\" (UniqueName: \"kubernetes.io/projected/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-kube-api-access-r88mm\") pod \"payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.486376 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.486203    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltnn4\" (UniqueName: \"kubernetes.io/projected/64c838bb-54de-491c-9a76-e7989c79f563-kube-api-access-ltnn4\") pod \"payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.486376 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.486251    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c838bb-54de-491c-9a76-e7989c79f563-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.486376 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.486285    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.486984 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.486965    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c838bb-54de-491c-9a76-e7989c79f563-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.495630 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.495601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltnn4\" (UniqueName: \"kubernetes.io/projected/64c838bb-54de-491c-9a76-e7989c79f563-kube-api-access-ltnn4\") pod \"payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.587270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.587185    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.587270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.587254    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r88mm\" (UniqueName: \"kubernetes.io/projected/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-kube-api-access-r88mm\") pod \"payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.588049 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.588029    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.596815 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.596782    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r88mm\" (UniqueName: \"kubernetes.io/projected/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-kube-api-access-r88mm\") pod \"payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.643547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.643510    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:13.701475 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.701442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:13.956144 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:13.956100    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"]
Sep 18 22:29:13.957197 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:13.957169    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c838bb_54de_491c_9a76_e7989c79f563.slice/crio-24204724c8f668a1ce9b8bea750b2bb3c1035fdad7a1c6e16681e24af40b35c7 WatchSource:0}: Error finding container 24204724c8f668a1ce9b8bea750b2bb3c1035fdad7a1c6e16681e24af40b35c7: Status 404 returned error can't find the container with id 24204724c8f668a1ce9b8bea750b2bb3c1035fdad7a1c6e16681e24af40b35c7
Sep 18 22:29:14.014167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.014135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"]
Sep 18 22:29:14.016564 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:14.016536    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d2b728_a8ff_4aea_8a07_52de0cb1e77e.slice/crio-2e6a0aeb2971436d18f5f09af8b94d554d35cdd1dd686f5907952bff3bceee44 WatchSource:0}: Error finding container 2e6a0aeb2971436d18f5f09af8b94d554d35cdd1dd686f5907952bff3bceee44: Status 404 returned error can't find the container with id 2e6a0aeb2971436d18f5f09af8b94d554d35cdd1dd686f5907952bff3bceee44
Sep 18 22:29:14.138992 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.138892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb" event={"ID":"61d2b728-a8ff-4aea-8a07-52de0cb1e77e","Type":"ContainerStarted","Data":"bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea"}
Sep 18 22:29:14.138992 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.138950    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb" event={"ID":"61d2b728-a8ff-4aea-8a07-52de0cb1e77e","Type":"ContainerStarted","Data":"2e6a0aeb2971436d18f5f09af8b94d554d35cdd1dd686f5907952bff3bceee44"}
Sep 18 22:29:14.139418 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.139270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:14.141256 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.141232    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg" event={"ID":"64c838bb-54de-491c-9a76-e7989c79f563","Type":"ContainerStarted","Data":"1b1f830ab6c0c562303a6e1cf5a7098dab5bf6019ebe8cf6c206fa828950cd46"}
Sep 18 22:29:14.141401 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.141275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg" event={"ID":"64c838bb-54de-491c-9a76-e7989c79f563","Type":"ContainerStarted","Data":"24204724c8f668a1ce9b8bea750b2bb3c1035fdad7a1c6e16681e24af40b35c7"}
Sep 18 22:29:14.141468 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.141443    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:14.156435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.156154    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb" podStartSLOduration=1.156138087 podStartE2EDuration="1.156138087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:14.155902383 +0000 UTC m=+738.516419241" watchObservedRunningTime="2026-09-18 22:29:14.156138087 +0000 UTC m=+738.516654945"
Sep 18 22:29:14.172825 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.172776    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg" podStartSLOduration=1.172759793 podStartE2EDuration="1.172759793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:14.171782843 +0000 UTC m=+738.532299726" watchObservedRunningTime="2026-09-18 22:29:14.172759793 +0000 UTC m=+738.533276702"
Sep 18 22:29:14.589553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.589515    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"]
Sep 18 22:29:14.593350 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.593332    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.595954 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.595934    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-c5fdd3-dockercfg-n7wcf\""
Sep 18 22:29:14.596090 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.595979    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-c5fdd3\""
Sep 18 22:29:14.605613 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.605590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"]
Sep 18 22:29:14.698108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.698079    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.698439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.698123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4c6v\" (UniqueName: \"kubernetes.io/projected/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-kube-api-access-z4c6v\") pod \"payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.799098 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.799069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.799254 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.799117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4c6v\" (UniqueName: \"kubernetes.io/projected/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-kube-api-access-z4c6v\") pod \"payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.799822 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.799804    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.811356 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.811330    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4c6v\" (UniqueName: \"kubernetes.io/projected/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-kube-api-access-z4c6v\") pod \"payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:14.905169 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:14.905134    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:15.148979 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:15.148939    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" event={"ID":"bbae3e8f-5a90-43cb-a953-c9adf8a65f53","Type":"ContainerStarted","Data":"01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316"}
Sep 18 22:29:15.206959 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:15.206886    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"]
Sep 18 22:29:15.267776 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:15.267722    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38f1bd2_99b8_4cd8_bbdf_8dcc3591e2b0.slice/crio-32a0ff9d9edc1dee46c252498e05719c36bd15606f75be26d9ff40059cb39429 WatchSource:0}: Error finding container 32a0ff9d9edc1dee46c252498e05719c36bd15606f75be26d9ff40059cb39429: Status 404 returned error can't find the container with id 32a0ff9d9edc1dee46c252498e05719c36bd15606f75be26d9ff40059cb39429
Sep 18 22:29:16.157641 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:16.157601    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc" event={"ID":"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0","Type":"ContainerStarted","Data":"59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7"}
Sep 18 22:29:16.158074 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:16.157667    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc" event={"ID":"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0","Type":"ContainerStarted","Data":"32a0ff9d9edc1dee46c252498e05719c36bd15606f75be26d9ff40059cb39429"}
Sep 18 22:29:16.158074 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:16.157912    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:16.158801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:16.158768    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:16.179905 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:16.179854    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc" podStartSLOduration=2.179837823 podStartE2EDuration="2.179837823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:16.178112964 +0000 UTC m=+740.538629833" watchObservedRunningTime="2026-09-18 22:29:16.179837823 +0000 UTC m=+740.540354680"
Sep 18 22:29:16.181077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:16.181043    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" podStartSLOduration=3.261759661 podStartE2EDuration="5.181027615s" totalImagesPullingTime="1.919267954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:15.263604346 +0000 UTC m=+739.624121215" watchObservedRunningTime="2026-09-18 22:29:16.181027615 +0000 UTC m=+740.541544472"
Sep 18 22:29:21.049978 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.049936    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"]
Sep 18 22:29:21.053747 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.053725    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.056414 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.056368    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-917b58\""
Sep 18 22:29:21.056629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.056610    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-917b58-dockercfg-v59mq\""
Sep 18 22:29:21.069778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.069749    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"]
Sep 18 22:29:21.113194 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.113153    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"]
Sep 18 22:29:21.116709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.116688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.134620 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.127877    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"]
Sep 18 22:29:21.159560 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.159521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-plugins-config-volume\") pod \"payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") " pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.159728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.159574    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5sqb\" (UniqueName: \"kubernetes.io/projected/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-kube-api-access-t5sqb\") pod \"payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.159728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.159666    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dws6g\" (UniqueName: \"kubernetes.io/projected/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-kube-api-access-dws6g\") pod \"payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") " pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.159728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.159708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.260445 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.260407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-plugins-config-volume\") pod \"payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") " pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.260601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.260464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5sqb\" (UniqueName: \"kubernetes.io/projected/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-kube-api-access-t5sqb\") pod \"payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.260601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.260494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dws6g\" (UniqueName: \"kubernetes.io/projected/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-kube-api-access-dws6g\") pod \"payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") " pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.260601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.260513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.261229 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.261207    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-plugins-config-volume\") pod \"payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") " pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.261271 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.261229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.269119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.269090    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5sqb\" (UniqueName: \"kubernetes.io/projected/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-kube-api-access-t5sqb\") pod \"payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.269279 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.269258    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dws6g\" (UniqueName: \"kubernetes.io/projected/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-kube-api-access-dws6g\") pod \"payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") " pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.363894 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.363810    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:21.435954 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.435922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:21.644045 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.644021    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"]
Sep 18 22:29:21.645263 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:21.645228    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047eceb7_dc09_4ae9_bcb6_0b7beee95cb9.slice/crio-6857ef716e1ccef29e413c04a7b8cc6fbfaa3c169e2dc729858981625f797c45 WatchSource:0}: Error finding container 6857ef716e1ccef29e413c04a7b8cc6fbfaa3c169e2dc729858981625f797c45: Status 404 returned error can't find the container with id 6857ef716e1ccef29e413c04a7b8cc6fbfaa3c169e2dc729858981625f797c45
Sep 18 22:29:21.711175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:21.711148    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"]
Sep 18 22:29:21.713138 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:21.713111    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e027dc_f87a_4671_bdc5_227ceaa9c9b6.slice/crio-d2481798812b50ea7c0730fa4a7bba26e037ae66b553f1f08b2005c897b80a3f WatchSource:0}: Error finding container d2481798812b50ea7c0730fa4a7bba26e037ae66b553f1f08b2005c897b80a3f: Status 404 returned error can't find the container with id d2481798812b50ea7c0730fa4a7bba26e037ae66b553f1f08b2005c897b80a3f
Sep 18 22:29:22.170277 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.170250    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:29:22.195749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.195716    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2" event={"ID":"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9","Type":"ContainerStarted","Data":"cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9"}
Sep 18 22:29:22.195937 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.195781    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:22.195937 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.195807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2" event={"ID":"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9","Type":"ContainerStarted","Data":"6857ef716e1ccef29e413c04a7b8cc6fbfaa3c169e2dc729858981625f797c45"}
Sep 18 22:29:22.197361 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.197334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk" event={"ID":"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6","Type":"ContainerStarted","Data":"793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854"}
Sep 18 22:29:22.197494 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.197411    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk" event={"ID":"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6","Type":"ContainerStarted","Data":"d2481798812b50ea7c0730fa4a7bba26e037ae66b553f1f08b2005c897b80a3f"}
Sep 18 22:29:22.197494 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.197441    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:22.219344 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.219283    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2" podStartSLOduration=1.21926004 podStartE2EDuration="1.21926004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:22.214864219 +0000 UTC m=+746.575381098" watchObservedRunningTime="2026-09-18 22:29:22.21926004 +0000 UTC m=+746.579776899"
Sep 18 22:29:22.235332 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:22.235275    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk" podStartSLOduration=1.235254218 podStartE2EDuration="1.235254218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:22.232566235 +0000 UTC m=+746.593083094" watchObservedRunningTime="2026-09-18 22:29:22.235254218 +0000 UTC m=+746.595771081"
Sep 18 22:29:24.143866 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:24.143833    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:29:24.144336 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:24.143919    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:29:25.151310 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:25.151281    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:29:26.160133 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:26.160102    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:29:27.167264 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:27.167237    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:29:33.205220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:33.205189    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:29:33.205610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:33.205261    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:29:38.690163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.690124    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"]
Sep 18 22:29:38.695662 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.695639    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.698320 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.698297    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-44d735-serving-cert\""
Sep 18 22:29:38.709493 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.709469    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"]
Sep 18 22:29:38.826724 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.826692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/118d4da0-e8ba-410e-9aff-57ae036f0657-openshift-service-ca\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.826919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.826731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nng6\" (UniqueName: \"kubernetes.io/projected/118d4da0-e8ba-410e-9aff-57ae036f0657-kube-api-access-8nng6\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.826919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.826762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.826919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.826811    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-metrics-certs\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.927745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.927708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/118d4da0-e8ba-410e-9aff-57ae036f0657-openshift-service-ca\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.927745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.927747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nng6\" (UniqueName: \"kubernetes.io/projected/118d4da0-e8ba-410e-9aff-57ae036f0657-kube-api-access-8nng6\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.927998 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.927768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.927998 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.927789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-metrics-certs\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.927998 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:29:38.927881    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-serving-cert: secret "maas-api-e2e-inf-b-44d735-serving-cert" not found
Sep 18 22:29:38.927998 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:29:38.927953    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls podName:118d4da0-e8ba-410e-9aff-57ae036f0657 nodeName:}" failed. No retries permitted until 2026-09-18 22:29:39.427934288 +0000 UTC m=+763.788451123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls") pod "maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" (UID: "118d4da0-e8ba-410e-9aff-57ae036f0657") : secret "maas-api-e2e-inf-b-44d735-serving-cert" not found
Sep 18 22:29:38.928504 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.928483    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/118d4da0-e8ba-410e-9aff-57ae036f0657-openshift-service-ca\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.930336 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.930316    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-metrics-certs\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:38.941638 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:38.941579    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nng6\" (UniqueName: \"kubernetes.io/projected/118d4da0-e8ba-410e-9aff-57ae036f0657-kube-api-access-8nng6\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:39.432574 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:39.432543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:39.435181 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:39.435155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls\") pod \"maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:39.608412 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:39.608358    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:39.891647 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:39.891619    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"]
Sep 18 22:29:39.892656 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:39.892635    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118d4da0_e8ba_410e_9aff_57ae036f0657.slice/crio-8b90fdfa00ad6c261b0ad71f8ad517bc75dedcbd51c53737cd882b6347f11f40 WatchSource:0}: Error finding container 8b90fdfa00ad6c261b0ad71f8ad517bc75dedcbd51c53737cd882b6347f11f40: Status 404 returned error can't find the container with id 8b90fdfa00ad6c261b0ad71f8ad517bc75dedcbd51c53737cd882b6347f11f40
Sep 18 22:29:40.303172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:40.303141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" event={"ID":"118d4da0-e8ba-410e-9aff-57ae036f0657","Type":"ContainerStarted","Data":"8b90fdfa00ad6c261b0ad71f8ad517bc75dedcbd51c53737cd882b6347f11f40"}
Sep 18 22:29:41.311010 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:41.310974    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" event={"ID":"118d4da0-e8ba-410e-9aff-57ae036f0657","Type":"ContainerStarted","Data":"8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d"}
Sep 18 22:29:41.311438 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:41.311077    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:41.329755 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:41.329700    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" podStartSLOduration=2.980772338 podStartE2EDuration="3.329683017s" totalImagesPullingTime="348.910679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:29:41.328347695 +0000 UTC m=+765.688864565" watchObservedRunningTime="2026-09-18 22:29:41.329683017 +0000 UTC m=+765.690199874"
Sep 18 22:29:43.560850 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.560804    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"]
Sep 18 22:29:43.571051 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.571016    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:43.574629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.574603    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8dgjh\""
Sep 18 22:29:43.600569 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.600541    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"]
Sep 18 22:29:43.673771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.673726    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe09510-139c-4e8f-ad83-3a2cf30bedd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-519912e2-wnhml\" (UID: \"5fe09510-139c-4e8f-ad83-3a2cf30bedd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:43.775312 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.775272    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe09510-139c-4e8f-ad83-3a2cf30bedd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-519912e2-wnhml\" (UID: \"5fe09510-139c-4e8f-ad83-3a2cf30bedd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:43.776126 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.776094    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe09510-139c-4e8f-ad83-3a2cf30bedd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-519912e2-wnhml\" (UID: \"5fe09510-139c-4e8f-ad83-3a2cf30bedd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:43.881394 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:43.881366    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:44.157857 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:44.157831    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"]
Sep 18 22:29:44.160023 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:29:44.159994    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe09510_139c_4e8f_ad83_3a2cf30bedd4.slice/crio-7644417d51c94b8e3bfbf4f0431e0edd7fc42c67ca41a929d45d3aa178dd0bed WatchSource:0}: Error finding container 7644417d51c94b8e3bfbf4f0431e0edd7fc42c67ca41a929d45d3aa178dd0bed: Status 404 returned error can't find the container with id 7644417d51c94b8e3bfbf4f0431e0edd7fc42c67ca41a929d45d3aa178dd0bed
Sep 18 22:29:44.328865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:44.328822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml" event={"ID":"5fe09510-139c-4e8f-ad83-3a2cf30bedd4","Type":"ContainerStarted","Data":"7644417d51c94b8e3bfbf4f0431e0edd7fc42c67ca41a929d45d3aa178dd0bed"}
Sep 18 22:29:47.322084 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:47.322059    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:29:47.352680 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:47.352634    2583 generic.go:356] "Generic (PLEG): container finished" podID="5fe09510-139c-4e8f-ad83-3a2cf30bedd4" containerID="2f4604b8eddc11f35f521ddf2920f740ed33a068204ab4325fcf6cc0f98ab4be" exitCode=0
Sep 18 22:29:47.352848 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:47.352708    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml" event={"ID":"5fe09510-139c-4e8f-ad83-3a2cf30bedd4","Type":"ContainerDied","Data":"2f4604b8eddc11f35f521ddf2920f740ed33a068204ab4325fcf6cc0f98ab4be"}
Sep 18 22:29:48.494265 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:48.494241    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:48.630093 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:48.630058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe09510-139c-4e8f-ad83-3a2cf30bedd4-maas-api-service-ca\") pod \"5fe09510-139c-4e8f-ad83-3a2cf30bedd4\" (UID: \"5fe09510-139c-4e8f-ad83-3a2cf30bedd4\") "
Sep 18 22:29:48.630468 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:48.630447    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe09510-139c-4e8f-ad83-3a2cf30bedd4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5fe09510-139c-4e8f-ad83-3a2cf30bedd4" (UID: "5fe09510-139c-4e8f-ad83-3a2cf30bedd4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:29:48.730726 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:48.730693    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe09510-139c-4e8f-ad83-3a2cf30bedd4-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:29:49.365878 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:49.365844    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"
Sep 18 22:29:49.366086 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:49.365888    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml" event={"ID":"5fe09510-139c-4e8f-ad83-3a2cf30bedd4","Type":"ContainerDied","Data":"7644417d51c94b8e3bfbf4f0431e0edd7fc42c67ca41a929d45d3aa178dd0bed"}
Sep 18 22:29:49.366086 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:49.365971    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7644417d51c94b8e3bfbf4f0431e0edd7fc42c67ca41a929d45d3aa178dd0bed"
Sep 18 22:29:55.748713 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:55.748674    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"]
Sep 18 22:29:55.751282 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:55.751259    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-519912e2-wnhml"]
Sep 18 22:29:57.683051 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:29:57.683019    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe09510-139c-4e8f-ad83-3a2cf30bedd4" path="/var/lib/kubelet/pods/5fe09510-139c-4e8f-ad83-3a2cf30bedd4/volumes"
Sep 18 22:30:00.152207 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.152167    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"]
Sep 18 22:30:00.152765 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.152745    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe09510-139c-4e8f-ad83-3a2cf30bedd4" containerName="revoke-keys"
Sep 18 22:30:00.152845 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.152769    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe09510-139c-4e8f-ad83-3a2cf30bedd4" containerName="revoke-keys"
Sep 18 22:30:00.152910 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.152897    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe09510-139c-4e8f-ad83-3a2cf30bedd4" containerName="revoke-keys"
Sep 18 22:30:00.157853 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.157831    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"]
Sep 18 22:30:00.157982 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.157945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"
Sep 18 22:30:00.161439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.161418    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8dgjh\""
Sep 18 22:30:00.161557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.161533    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"
Sep 18 22:30:00.169541 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.169518    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"]
Sep 18 22:30:00.169651 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.169592    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"
Sep 18 22:30:00.179078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.179054    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"
Sep 18 22:30:00.183907 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.183884    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"]
Sep 18 22:30:00.472034 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.472010    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"]
Sep 18 22:30:00.473007 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:30:00.472984    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd4d959_1ca7_42be_962d_6b3e4b1b7fc1.slice/crio-78c562b3e55a04e5e7ef432a5a581a0867f2d177a4260155344903f281e70076 WatchSource:0}: Error finding container 78c562b3e55a04e5e7ef432a5a581a0867f2d177a4260155344903f281e70076: Status 404 returned error can't find the container with id 78c562b3e55a04e5e7ef432a5a581a0867f2d177a4260155344903f281e70076
Sep 18 22:30:00.474535 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:30:00.474507    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5465553c_3f51_4668_bd8e_5f100cd66f61.slice/crio-d5365c4f73f4fafae86c3c1203afbcd8086f910c8414c236ae250be8a6132596 WatchSource:0}: Error finding container d5365c4f73f4fafae86c3c1203afbcd8086f910c8414c236ae250be8a6132596: Status 404 returned error can't find the container with id d5365c4f73f4fafae86c3c1203afbcd8086f910c8414c236ae250be8a6132596
Sep 18 22:30:00.476012 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:00.475983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"]
Sep 18 22:30:01.438742 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:01.438710    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6" event={"ID":"5465553c-3f51-4668-bd8e-5f100cd66f61","Type":"ContainerStarted","Data":"e672df7a5266d71534cb6cf4cd8648fde916c97ec89317efbd7ca38eeb6fb6eb"}
Sep 18 22:30:01.439147 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:01.438771    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6" event={"ID":"5465553c-3f51-4668-bd8e-5f100cd66f61","Type":"ContainerStarted","Data":"d5365c4f73f4fafae86c3c1203afbcd8086f910c8414c236ae250be8a6132596"}
Sep 18 22:30:01.440450 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:01.440424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf" event={"ID":"ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1","Type":"ContainerStarted","Data":"b286ca71062857eb77dcfa94121ad5b00ed30d35aa99470cb618aad4d67d06d4"}
Sep 18 22:30:01.440581 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:01.440470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf" event={"ID":"ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1","Type":"ContainerStarted","Data":"78c562b3e55a04e5e7ef432a5a581a0867f2d177a4260155344903f281e70076"}
Sep 18 22:30:01.455282 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:01.455229    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6" podStartSLOduration=1.455216385 podStartE2EDuration="1.455216385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:30:01.452925556 +0000 UTC m=+785.813442414" watchObservedRunningTime="2026-09-18 22:30:01.455216385 +0000 UTC m=+785.815733241"
Sep 18 22:30:01.468943 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:01.468893    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf" podStartSLOduration=1.468876121 podStartE2EDuration="1.468876121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:30:01.467588658 +0000 UTC m=+785.828105520" watchObservedRunningTime="2026-09-18 22:30:01.468876121 +0000 UTC m=+785.829393019"
Sep 18 22:30:02.448134 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:02.448098    2583 generic.go:356] "Generic (PLEG): container finished" podID="5465553c-3f51-4668-bd8e-5f100cd66f61" containerID="e672df7a5266d71534cb6cf4cd8648fde916c97ec89317efbd7ca38eeb6fb6eb" exitCode=0
Sep 18 22:30:02.448636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:02.448183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6" event={"ID":"5465553c-3f51-4668-bd8e-5f100cd66f61","Type":"ContainerDied","Data":"e672df7a5266d71534cb6cf4cd8648fde916c97ec89317efbd7ca38eeb6fb6eb"}
Sep 18 22:30:02.450002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:02.449967    2583 generic.go:356] "Generic (PLEG): container finished" podID="ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1" containerID="b286ca71062857eb77dcfa94121ad5b00ed30d35aa99470cb618aad4d67d06d4" exitCode=0
Sep 18 22:30:02.450135 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:02.450016    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf" event={"ID":"ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1","Type":"ContainerDied","Data":"b286ca71062857eb77dcfa94121ad5b00ed30d35aa99470cb618aad4d67d06d4"}
Sep 18 22:30:03.436517 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.436479    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"]
Sep 18 22:30:03.440079 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.440050    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.443128 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.443092    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-67d98638-dockercfg-d8r2s\""
Sep 18 22:30:03.443258 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.443157    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-67d98638\""
Sep 18 22:30:03.453758 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.453733    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"]
Sep 18 22:30:03.465675 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.465513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-plugins-config-volume\") pod \"payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") " pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.466081 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.465750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62567\" (UniqueName: \"kubernetes.io/projected/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-kube-api-access-62567\") pod \"payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") " pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.567252 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.567214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62567\" (UniqueName: \"kubernetes.io/projected/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-kube-api-access-62567\") pod \"payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") " pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.567495 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.567312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-plugins-config-volume\") pod \"payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") " pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.568296 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.568257    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-plugins-config-volume\") pod \"payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") " pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.578128 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.578098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62567\" (UniqueName: \"kubernetes.io/projected/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-kube-api-access-62567\") pod \"payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") " pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:03.602303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.602281    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"
Sep 18 22:30:03.633312 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.633279    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"
Sep 18 22:30:03.751326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:03.751289    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:04.036106 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.036078    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"]
Sep 18 22:30:04.037099 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:30:04.037069    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4fb0c9_5fc2_46a5_a364_aa6c5820d014.slice/crio-1bf48d81e9a34febd8b5be75b159d2a03134044b44158d30fe3f5d556255c73a WatchSource:0}: Error finding container 1bf48d81e9a34febd8b5be75b159d2a03134044b44158d30fe3f5d556255c73a: Status 404 returned error can't find the container with id 1bf48d81e9a34febd8b5be75b159d2a03134044b44158d30fe3f5d556255c73a
Sep 18 22:30:04.464467 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.464417    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs" event={"ID":"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014","Type":"ContainerStarted","Data":"9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3"}
Sep 18 22:30:04.464931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.464484    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs" event={"ID":"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014","Type":"ContainerStarted","Data":"1bf48d81e9a34febd8b5be75b159d2a03134044b44158d30fe3f5d556255c73a"}
Sep 18 22:30:04.464931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.464515    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:04.465843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.465822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6" event={"ID":"5465553c-3f51-4668-bd8e-5f100cd66f61","Type":"ContainerDied","Data":"d5365c4f73f4fafae86c3c1203afbcd8086f910c8414c236ae250be8a6132596"}
Sep 18 22:30:04.465983 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.465865    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5365c4f73f4fafae86c3c1203afbcd8086f910c8414c236ae250be8a6132596"
Sep 18 22:30:04.465983 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.465826    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"
Sep 18 22:30:04.467378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.467355    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"
Sep 18 22:30:04.467498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.467423    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf" event={"ID":"ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1","Type":"ContainerDied","Data":"78c562b3e55a04e5e7ef432a5a581a0867f2d177a4260155344903f281e70076"}
Sep 18 22:30:04.467498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.467467    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78c562b3e55a04e5e7ef432a5a581a0867f2d177a4260155344903f281e70076"
Sep 18 22:30:04.486242 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:04.486192    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs" podStartSLOduration=1.486172764 podStartE2EDuration="1.486172764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:30:04.484129141 +0000 UTC m=+788.844645998" watchObservedRunningTime="2026-09-18 22:30:04.486172764 +0000 UTC m=+788.846689628"
Sep 18 22:30:15.476600 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:15.476527    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:40.781617 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:40.781583    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"]
Sep 18 22:30:40.782056 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:40.781806    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs" podUID="bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" containerName="payload-processing" containerID="cri-o://9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3" gracePeriod=30
Sep 18 22:30:41.056264 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.055680    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:41.114548 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.114519    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-plugins-config-volume\") pod \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") "
Sep 18 22:30:41.114703 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.114591    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62567\" (UniqueName: \"kubernetes.io/projected/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-kube-api-access-62567\") pod \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\" (UID: \"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014\") "
Sep 18 22:30:41.114891 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.114868    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" (UID: "bf4fb0c9-5fc2-46a5-a364-aa6c5820d014"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:30:41.116836 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.116806    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-kube-api-access-62567" (OuterVolumeSpecName: "kube-api-access-62567") pod "bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" (UID: "bf4fb0c9-5fc2-46a5-a364-aa6c5820d014"). InnerVolumeSpecName "kube-api-access-62567". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:30:41.216355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.216324    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62567\" (UniqueName: \"kubernetes.io/projected/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-kube-api-access-62567\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:30:41.216355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.216354    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:30:41.687377 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.687347    2583 generic.go:356] "Generic (PLEG): container finished" podID="bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" containerID="9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3" exitCode=0
Sep 18 22:30:41.687553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.687453    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"
Sep 18 22:30:41.687553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.687491    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs" event={"ID":"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014","Type":"ContainerDied","Data":"9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3"}
Sep 18 22:30:41.687553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.687536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs" event={"ID":"bf4fb0c9-5fc2-46a5-a364-aa6c5820d014","Type":"ContainerDied","Data":"1bf48d81e9a34febd8b5be75b159d2a03134044b44158d30fe3f5d556255c73a"}
Sep 18 22:30:41.687654 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.687558    2583 scope.go:117] "RemoveContainer" containerID="9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3"
Sep 18 22:30:41.697914 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.697896    2583 scope.go:117] "RemoveContainer" containerID="9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3"
Sep 18 22:30:41.698221 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:30:41.698199    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3\": container with ID starting with 9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3 not found: ID does not exist" containerID="9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3"
Sep 18 22:30:41.698275 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.698230    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3"} err="failed to get container status \"9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3\": rpc error: code = NotFound desc = could not find container \"9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3\": container with ID starting with 9f3508673dc285b502750b40e3437e31f53759b387d6f277d51680574cfb4ab3 not found: ID does not exist"
Sep 18 22:30:41.707755 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.707731    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"]
Sep 18 22:30:41.711167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:41.711143    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-67d98638-7597c7d646-9mwgs"]
Sep 18 22:30:43.683112 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:43.683067    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" path="/var/lib/kubelet/pods/bf4fb0c9-5fc2-46a5-a364-aa6c5820d014/volumes"
Sep 18 22:30:47.506267 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506225    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"]
Sep 18 22:30:47.506889 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506867    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1" containerName="cleanup"
Sep 18 22:30:47.506973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506893    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1" containerName="cleanup"
Sep 18 22:30:47.506973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506905    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5465553c-3f51-4668-bd8e-5f100cd66f61" containerName="cleanup"
Sep 18 22:30:47.506973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506913    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5465553c-3f51-4668-bd8e-5f100cd66f61" containerName="cleanup"
Sep 18 22:30:47.506973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506931    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" containerName="payload-processing"
Sep 18 22:30:47.506973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.506942    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" containerName="payload-processing"
Sep 18 22:30:47.507206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.507048    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1" containerName="cleanup"
Sep 18 22:30:47.507206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.507061    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4fb0c9-5fc2-46a5-a364-aa6c5820d014" containerName="payload-processing"
Sep 18 22:30:47.507206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.507074    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5465553c-3f51-4668-bd8e-5f100cd66f61" containerName="cleanup"
Sep 18 22:30:47.510523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.510502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.515442 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.515423    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9d0906fb\""
Sep 18 22:30:47.519023 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.519003    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9d0906fb-dockercfg-k6596\""
Sep 18 22:30:47.529979 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.529956    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"]
Sep 18 22:30:47.583872 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.583841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8d6b\" (UniqueName: \"kubernetes.io/projected/32765727-ebb2-40a5-a3de-0f64deb2b12b-kube-api-access-w8d6b\") pod \"payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") " pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.584017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.583933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32765727-ebb2-40a5-a3de-0f64deb2b12b-plugins-config-volume\") pod \"payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") " pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.685255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.685221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32765727-ebb2-40a5-a3de-0f64deb2b12b-plugins-config-volume\") pod \"payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") " pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.685469 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.685288    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8d6b\" (UniqueName: \"kubernetes.io/projected/32765727-ebb2-40a5-a3de-0f64deb2b12b-kube-api-access-w8d6b\") pod \"payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") " pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.686195 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.686169    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32765727-ebb2-40a5-a3de-0f64deb2b12b-plugins-config-volume\") pod \"payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") " pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.694180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.694159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8d6b\" (UniqueName: \"kubernetes.io/projected/32765727-ebb2-40a5-a3de-0f64deb2b12b-kube-api-access-w8d6b\") pod \"payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") " pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:47.820954 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:47.820877    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:48.105244 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:48.105217    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"]
Sep 18 22:30:48.105789 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:30:48.105763    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32765727_ebb2_40a5_a3de_0f64deb2b12b.slice/crio-fea57772629f42da7f5b639a2f9eb4ddc50307aece083181e2db2a2f1612170d WatchSource:0}: Error finding container fea57772629f42da7f5b639a2f9eb4ddc50307aece083181e2db2a2f1612170d: Status 404 returned error can't find the container with id fea57772629f42da7f5b639a2f9eb4ddc50307aece083181e2db2a2f1612170d
Sep 18 22:30:48.732449 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:48.732379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp" event={"ID":"32765727-ebb2-40a5-a3de-0f64deb2b12b","Type":"ContainerStarted","Data":"3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce"}
Sep 18 22:30:48.732931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:48.732462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp" event={"ID":"32765727-ebb2-40a5-a3de-0f64deb2b12b","Type":"ContainerStarted","Data":"fea57772629f42da7f5b639a2f9eb4ddc50307aece083181e2db2a2f1612170d"}
Sep 18 22:30:48.732931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:48.732483    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:30:48.800921 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:48.800866    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp" podStartSLOduration=1.8008507630000001 podStartE2EDuration="1.800850763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:30:48.794095813 +0000 UTC m=+833.154612669" watchObservedRunningTime="2026-09-18 22:30:48.800850763 +0000 UTC m=+833.161367619"
Sep 18 22:30:59.740259 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:30:59.740232    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:31:15.558895 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.558859    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"]
Sep 18 22:31:15.562461 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.562443    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:15.564807 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.564788    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8dgjh\""
Sep 18 22:31:15.569551 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.569524    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"]
Sep 18 22:31:15.647720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.647686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8375f29-2a43-4f00-8df6-311098f25110-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8\" (UID: \"e8375f29-2a43-4f00-8df6-311098f25110\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:15.749070 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.749037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8375f29-2a43-4f00-8df6-311098f25110-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8\" (UID: \"e8375f29-2a43-4f00-8df6-311098f25110\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:15.749690 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.749672    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8375f29-2a43-4f00-8df6-311098f25110-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8\" (UID: \"e8375f29-2a43-4f00-8df6-311098f25110\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:15.874845 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:15.874766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:16.156115 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:16.156084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"]
Sep 18 22:31:16.157926 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:31:16.157891    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8375f29_2a43_4f00_8df6_311098f25110.slice/crio-0aeb1eebff50859aa8a9fc848ca60c19c080b4c4088540f751104be2da32192d WatchSource:0}: Error finding container 0aeb1eebff50859aa8a9fc848ca60c19c080b4c4088540f751104be2da32192d: Status 404 returned error can't find the container with id 0aeb1eebff50859aa8a9fc848ca60c19c080b4c4088540f751104be2da32192d
Sep 18 22:31:16.894677 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:16.894648    2583 generic.go:356] "Generic (PLEG): container finished" podID="e8375f29-2a43-4f00-8df6-311098f25110" containerID="f0189af619c954c68ee8982eb9b029d3aa9da5b2623d1e22b43355af108a179a" exitCode=0
Sep 18 22:31:16.895049 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:16.894725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8" event={"ID":"e8375f29-2a43-4f00-8df6-311098f25110","Type":"ContainerDied","Data":"f0189af619c954c68ee8982eb9b029d3aa9da5b2623d1e22b43355af108a179a"}
Sep 18 22:31:16.895049 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:16.894774    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8" event={"ID":"e8375f29-2a43-4f00-8df6-311098f25110","Type":"ContainerStarted","Data":"0aeb1eebff50859aa8a9fc848ca60c19c080b4c4088540f751104be2da32192d"}
Sep 18 22:31:18.039206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.039183    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:18.070668 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.070643    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8375f29-2a43-4f00-8df6-311098f25110-maas-api-service-ca\") pod \"e8375f29-2a43-4f00-8df6-311098f25110\" (UID: \"e8375f29-2a43-4f00-8df6-311098f25110\") "
Sep 18 22:31:18.070970 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.070948    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8375f29-2a43-4f00-8df6-311098f25110-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8375f29-2a43-4f00-8df6-311098f25110" (UID: "e8375f29-2a43-4f00-8df6-311098f25110"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:31:18.171868 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.171789    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8375f29-2a43-4f00-8df6-311098f25110-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:18.914637 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.914609    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"
Sep 18 22:31:18.914637 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.914627    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8" event={"ID":"e8375f29-2a43-4f00-8df6-311098f25110","Type":"ContainerDied","Data":"0aeb1eebff50859aa8a9fc848ca60c19c080b4c4088540f751104be2da32192d"}
Sep 18 22:31:18.914936 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:18.914708    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0aeb1eebff50859aa8a9fc848ca60c19c080b4c4088540f751104be2da32192d"
Sep 18 22:31:27.226840 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.226801    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"]
Sep 18 22:31:27.227218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.227060    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp" podUID="32765727-ebb2-40a5-a3de-0f64deb2b12b" containerName="payload-processing" containerID="cri-o://3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce" gracePeriod=30
Sep 18 22:31:27.389862 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.389817    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"]
Sep 18 22:31:27.393217 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.393191    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9d0906fb-dxth8"]
Sep 18 22:31:27.491479 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.491415    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:31:27.566706 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.566675    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8d6b\" (UniqueName: \"kubernetes.io/projected/32765727-ebb2-40a5-a3de-0f64deb2b12b-kube-api-access-w8d6b\") pod \"32765727-ebb2-40a5-a3de-0f64deb2b12b\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") "
Sep 18 22:31:27.566879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.566726    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32765727-ebb2-40a5-a3de-0f64deb2b12b-plugins-config-volume\") pod \"32765727-ebb2-40a5-a3de-0f64deb2b12b\" (UID: \"32765727-ebb2-40a5-a3de-0f64deb2b12b\") "
Sep 18 22:31:27.567143 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.567116    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32765727-ebb2-40a5-a3de-0f64deb2b12b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32765727-ebb2-40a5-a3de-0f64deb2b12b" (UID: "32765727-ebb2-40a5-a3de-0f64deb2b12b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:31:27.568767 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.568746    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32765727-ebb2-40a5-a3de-0f64deb2b12b-kube-api-access-w8d6b" (OuterVolumeSpecName: "kube-api-access-w8d6b") pod "32765727-ebb2-40a5-a3de-0f64deb2b12b" (UID: "32765727-ebb2-40a5-a3de-0f64deb2b12b"). InnerVolumeSpecName "kube-api-access-w8d6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:31:27.667336 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.667298    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8d6b\" (UniqueName: \"kubernetes.io/projected/32765727-ebb2-40a5-a3de-0f64deb2b12b-kube-api-access-w8d6b\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:27.667336 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.667328    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32765727-ebb2-40a5-a3de-0f64deb2b12b-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:27.683396 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.683362    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8375f29-2a43-4f00-8df6-311098f25110" path="/var/lib/kubelet/pods/e8375f29-2a43-4f00-8df6-311098f25110/volumes"
Sep 18 22:31:27.974192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.974159    2583 generic.go:356] "Generic (PLEG): container finished" podID="32765727-ebb2-40a5-a3de-0f64deb2b12b" containerID="3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce" exitCode=0
Sep 18 22:31:27.974411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.974211    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"
Sep 18 22:31:27.974411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.974231    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp" event={"ID":"32765727-ebb2-40a5-a3de-0f64deb2b12b","Type":"ContainerDied","Data":"3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce"}
Sep 18 22:31:27.974411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.974278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp" event={"ID":"32765727-ebb2-40a5-a3de-0f64deb2b12b","Type":"ContainerDied","Data":"fea57772629f42da7f5b639a2f9eb4ddc50307aece083181e2db2a2f1612170d"}
Sep 18 22:31:27.974411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.974301    2583 scope.go:117] "RemoveContainer" containerID="3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce"
Sep 18 22:31:27.983734 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.983716    2583 scope.go:117] "RemoveContainer" containerID="3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce"
Sep 18 22:31:27.984031 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:31:27.984011    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce\": container with ID starting with 3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce not found: ID does not exist" containerID="3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce"
Sep 18 22:31:27.984087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.984040    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce"} err="failed to get container status \"3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce\": rpc error: code = NotFound desc = could not find container \"3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce\": container with ID starting with 3ffce11fc11b89421cd7b7e301b22d05000a531a5a7da7c0aca4bea84d37b2ce not found: ID does not exist"
Sep 18 22:31:27.992370 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.992342    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"]
Sep 18 22:31:27.998579 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:27.998557    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9d0906fb-96fc4b7-j5xgp"]
Sep 18 22:31:29.683280 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:29.683241    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32765727-ebb2-40a5-a3de-0f64deb2b12b" path="/var/lib/kubelet/pods/32765727-ebb2-40a5-a3de-0f64deb2b12b/volumes"
Sep 18 22:31:36.099087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.099047    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"]
Sep 18 22:31:36.099920 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.099877    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2" podUID="047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" containerName="payload-processing" containerID="cri-o://cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9" gracePeriod=30
Sep 18 22:31:36.114283 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.114250    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"]
Sep 18 22:31:36.114588 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.114540    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk" podUID="a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" containerName="payload-pre-processing" containerID="cri-o://793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854" gracePeriod=30
Sep 18 22:31:36.370071 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.370044    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:31:36.417980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.417952    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:31:36.450357 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.450324    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dws6g\" (UniqueName: \"kubernetes.io/projected/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-kube-api-access-dws6g\") pod \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") "
Sep 18 22:31:36.450543 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.450488    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-plugins-config-volume\") pod \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\" (UID: \"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9\") "
Sep 18 22:31:36.450862 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.450835    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" (UID: "047eceb7-dc09-4ae9-bcb6-0b7beee95cb9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:31:36.452472 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.452453    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-kube-api-access-dws6g" (OuterVolumeSpecName: "kube-api-access-dws6g") pod "047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" (UID: "047eceb7-dc09-4ae9-bcb6-0b7beee95cb9"). InnerVolumeSpecName "kube-api-access-dws6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:31:36.551854 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.551809    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-plugins-config-volume\") pod \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") "
Sep 18 22:31:36.552073 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.551879    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5sqb\" (UniqueName: \"kubernetes.io/projected/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-kube-api-access-t5sqb\") pod \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\" (UID: \"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6\") "
Sep 18 22:31:36.552156 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.552129    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dws6g\" (UniqueName: \"kubernetes.io/projected/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-kube-api-access-dws6g\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:36.552156 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.552151    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:36.552411 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.552356    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" (UID: "a7e027dc-f87a-4671-bdc5-227ceaa9c9b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:31:36.554357 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.554326    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-kube-api-access-t5sqb" (OuterVolumeSpecName: "kube-api-access-t5sqb") pod "a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" (UID: "a7e027dc-f87a-4671-bdc5-227ceaa9c9b6"). InnerVolumeSpecName "kube-api-access-t5sqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:31:36.653005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.652970    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:36.653005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:36.653003    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5sqb\" (UniqueName: \"kubernetes.io/projected/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6-kube-api-access-t5sqb\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:31:37.031719 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.031626    2583 generic.go:356] "Generic (PLEG): container finished" podID="a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" containerID="793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854" exitCode=0
Sep 18 22:31:37.031719 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.031672    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"
Sep 18 22:31:37.031953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.031714    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk" event={"ID":"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6","Type":"ContainerDied","Data":"793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854"}
Sep 18 22:31:37.031953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.031763    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk" event={"ID":"a7e027dc-f87a-4671-bdc5-227ceaa9c9b6","Type":"ContainerDied","Data":"d2481798812b50ea7c0730fa4a7bba26e037ae66b553f1f08b2005c897b80a3f"}
Sep 18 22:31:37.031953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.031781    2583 scope.go:117] "RemoveContainer" containerID="793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854"
Sep 18 22:31:37.033144 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.033117    2583 generic.go:356] "Generic (PLEG): container finished" podID="047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" containerID="cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9" exitCode=0
Sep 18 22:31:37.033239 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.033150    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2" event={"ID":"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9","Type":"ContainerDied","Data":"cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9"}
Sep 18 22:31:37.033239 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.033174    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"
Sep 18 22:31:37.033239 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.033206    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2" event={"ID":"047eceb7-dc09-4ae9-bcb6-0b7beee95cb9","Type":"ContainerDied","Data":"6857ef716e1ccef29e413c04a7b8cc6fbfaa3c169e2dc729858981625f797c45"}
Sep 18 22:31:37.041917 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.041897    2583 scope.go:117] "RemoveContainer" containerID="793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854"
Sep 18 22:31:37.042158 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:31:37.042141    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854\": container with ID starting with 793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854 not found: ID does not exist" containerID="793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854"
Sep 18 22:31:37.042218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.042166    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854"} err="failed to get container status \"793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854\": rpc error: code = NotFound desc = could not find container \"793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854\": container with ID starting with 793806cebc63dec729994bbc1a47202c0f0dadb6e845b2dcaf0a90625de34854 not found: ID does not exist"
Sep 18 22:31:37.042218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.042182    2583 scope.go:117] "RemoveContainer" containerID="cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9"
Sep 18 22:31:37.050716 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.050698    2583 scope.go:117] "RemoveContainer" containerID="cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9"
Sep 18 22:31:37.050967 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:31:37.050951    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9\": container with ID starting with cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9 not found: ID does not exist" containerID="cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9"
Sep 18 22:31:37.051016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.050973    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9"} err="failed to get container status \"cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9\": rpc error: code = NotFound desc = could not find container \"cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9\": container with ID starting with cb7559c15927259e9b7e295d40f9c1f6a7f883cdf79a2bc9c2af3f8c032a2cb9 not found: ID does not exist"
Sep 18 22:31:37.062835 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.062737    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"]
Sep 18 22:31:37.066115 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.066089    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-917b58-879dfc946-rz6sk"]
Sep 18 22:31:37.077440 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.077416    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"]
Sep 18 22:31:37.083927 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.083903    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-917b58-757bf8cb8f-j5qj2"]
Sep 18 22:31:37.312786 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.312709    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"]
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313114    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32765727-ebb2-40a5-a3de-0f64deb2b12b" containerName="payload-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313125    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="32765727-ebb2-40a5-a3de-0f64deb2b12b" containerName="payload-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313133    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8375f29-2a43-4f00-8df6-311098f25110" containerName="revoke-keys"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313138    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8375f29-2a43-4f00-8df6-311098f25110" containerName="revoke-keys"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313151    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" containerName="payload-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313158    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" containerName="payload-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313170    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" containerName="payload-pre-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313175    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" containerName="payload-pre-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313230    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="32765727-ebb2-40a5-a3de-0f64deb2b12b" containerName="payload-processing"
Sep 18 22:31:37.313233 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313239    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" containerName="payload-processing"
Sep 18 22:31:37.313573 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313244    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" containerName="payload-pre-processing"
Sep 18 22:31:37.313573 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.313251    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8375f29-2a43-4f00-8df6-311098f25110" containerName="revoke-keys"
Sep 18 22:31:37.317878 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.317855    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.320647 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.320617    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-49880939-dockercfg-hcrrv\""
Sep 18 22:31:37.320773 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.320659    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-49880939\""
Sep 18 22:31:37.330329 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.330301    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"]
Sep 18 22:31:37.333899 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.333879    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"]
Sep 18 22:31:37.333997 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.333980    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.348085 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.348060    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"]
Sep 18 22:31:37.460023 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.459990    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629b85e4-bb48-4966-9a47-186df168996f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.460023 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.460035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcnb4\" (UniqueName: \"kubernetes.io/projected/629b85e4-bb48-4966-9a47-186df168996f-kube-api-access-lcnb4\") pod \"payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.460241 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.460142    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82f9253-cf6f-40f4-a639-bb89d6169b14-plugins-config-volume\") pod \"payload-processing-e2e-derive-49880939-7587875649-nffhw\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") " pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.460241 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.460203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpncx\" (UniqueName: \"kubernetes.io/projected/b82f9253-cf6f-40f4-a639-bb89d6169b14-kube-api-access-cpncx\") pod \"payload-processing-e2e-derive-49880939-7587875649-nffhw\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") " pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.560816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.560780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpncx\" (UniqueName: \"kubernetes.io/projected/b82f9253-cf6f-40f4-a639-bb89d6169b14-kube-api-access-cpncx\") pod \"payload-processing-e2e-derive-49880939-7587875649-nffhw\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") " pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.561016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.560836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629b85e4-bb48-4966-9a47-186df168996f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.561016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.560862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcnb4\" (UniqueName: \"kubernetes.io/projected/629b85e4-bb48-4966-9a47-186df168996f-kube-api-access-lcnb4\") pod \"payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.561016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.560905    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82f9253-cf6f-40f4-a639-bb89d6169b14-plugins-config-volume\") pod \"payload-processing-e2e-derive-49880939-7587875649-nffhw\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") " pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.561799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.561775    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82f9253-cf6f-40f4-a639-bb89d6169b14-plugins-config-volume\") pod \"payload-processing-e2e-derive-49880939-7587875649-nffhw\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") " pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.561799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.561790    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629b85e4-bb48-4966-9a47-186df168996f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.573350 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.573287    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpncx\" (UniqueName: \"kubernetes.io/projected/b82f9253-cf6f-40f4-a639-bb89d6169b14-kube-api-access-cpncx\") pod \"payload-processing-e2e-derive-49880939-7587875649-nffhw\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") " pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.573447 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.573420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcnb4\" (UniqueName: \"kubernetes.io/projected/629b85e4-bb48-4966-9a47-186df168996f-kube-api-access-lcnb4\") pod \"payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.629487 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.629447    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:37.644735 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.644709    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:37.686984 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.686747    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="047eceb7-dc09-4ae9-bcb6-0b7beee95cb9" path="/var/lib/kubelet/pods/047eceb7-dc09-4ae9-bcb6-0b7beee95cb9/volumes"
Sep 18 22:31:37.687305 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.687280    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e027dc-f87a-4671-bdc5-227ceaa9c9b6" path="/var/lib/kubelet/pods/a7e027dc-f87a-4671-bdc5-227ceaa9c9b6/volumes"
Sep 18 22:31:37.917583 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.917557    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"]
Sep 18 22:31:37.919497 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:31:37.919462    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629b85e4_bb48_4966_9a47_186df168996f.slice/crio-1179fb363de4283252c7ce514d566fae3ebf00a00b699efc3b2ba20e54ab987e WatchSource:0}: Error finding container 1179fb363de4283252c7ce514d566fae3ebf00a00b699efc3b2ba20e54ab987e: Status 404 returned error can't find the container with id 1179fb363de4283252c7ce514d566fae3ebf00a00b699efc3b2ba20e54ab987e
Sep 18 22:31:37.948120 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:37.948096    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"]
Sep 18 22:31:37.949112 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:31:37.949087    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82f9253_cf6f_40f4_a639_bb89d6169b14.slice/crio-ccf2f3ec27da61478a9d0aba9b00f606d98297b22e36fe2e64cdfb6c881963fb WatchSource:0}: Error finding container ccf2f3ec27da61478a9d0aba9b00f606d98297b22e36fe2e64cdfb6c881963fb: Status 404 returned error can't find the container with id ccf2f3ec27da61478a9d0aba9b00f606d98297b22e36fe2e64cdfb6c881963fb
Sep 18 22:31:38.042079 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.042044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw" event={"ID":"b82f9253-cf6f-40f4-a639-bb89d6169b14","Type":"ContainerStarted","Data":"42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa"}
Sep 18 22:31:38.042251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.042103    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw" event={"ID":"b82f9253-cf6f-40f4-a639-bb89d6169b14","Type":"ContainerStarted","Data":"ccf2f3ec27da61478a9d0aba9b00f606d98297b22e36fe2e64cdfb6c881963fb"}
Sep 18 22:31:38.042251 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.042137    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:38.045969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.045943    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f" event={"ID":"629b85e4-bb48-4966-9a47-186df168996f","Type":"ContainerStarted","Data":"3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4"}
Sep 18 22:31:38.046108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.045991    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:38.046108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.046007    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f" event={"ID":"629b85e4-bb48-4966-9a47-186df168996f","Type":"ContainerStarted","Data":"1179fb363de4283252c7ce514d566fae3ebf00a00b699efc3b2ba20e54ab987e"}
Sep 18 22:31:38.066306 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.066247    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw" podStartSLOduration=1.066224855 podStartE2EDuration="1.066224855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:31:38.062485276 +0000 UTC m=+882.423002124" watchObservedRunningTime="2026-09-18 22:31:38.066224855 +0000 UTC m=+882.426741721"
Sep 18 22:31:38.089703 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:38.089646    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f" podStartSLOduration=1.089625636 podStartE2EDuration="1.089625636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:31:38.086346241 +0000 UTC m=+882.446863099" watchObservedRunningTime="2026-09-18 22:31:38.089625636 +0000 UTC m=+882.450142494"
Sep 18 22:31:49.055296 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:49.055201    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:31:49.055762 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:49.055382    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:31:57.624592 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:57.624565    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:31:57.625132 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:31:57.625112    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:32:07.053953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.053914    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"]
Sep 18 22:32:07.058963 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.058944    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:07.061486 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.061463    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8dgjh\""
Sep 18 22:32:07.066637 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.066609    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"]
Sep 18 22:32:07.118869 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.118832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf0ab99-2644-4647-a591-11789541c7f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-49880939-pxw4v\" (UID: \"8cf0ab99-2644-4647-a591-11789541c7f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:07.219766 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.219734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf0ab99-2644-4647-a591-11789541c7f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-49880939-pxw4v\" (UID: \"8cf0ab99-2644-4647-a591-11789541c7f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:07.220400 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.220369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf0ab99-2644-4647-a591-11789541c7f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-49880939-pxw4v\" (UID: \"8cf0ab99-2644-4647-a591-11789541c7f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:07.372527 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.372441    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:07.653020 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:07.652983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"]
Sep 18 22:32:07.654063 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:32:07.654027    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf0ab99_2644_4647_a591_11789541c7f3.slice/crio-c09d9a2ecf848b068f33b003590c222324c7c1b5150cf765328d4bdbcc9eafaa WatchSource:0}: Error finding container c09d9a2ecf848b068f33b003590c222324c7c1b5150cf765328d4bdbcc9eafaa: Status 404 returned error can't find the container with id c09d9a2ecf848b068f33b003590c222324c7c1b5150cf765328d4bdbcc9eafaa
Sep 18 22:32:08.222129 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:08.222039    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v" event={"ID":"8cf0ab99-2644-4647-a591-11789541c7f3","Type":"ContainerStarted","Data":"bbb87f6ee2c66c92c8bd04114947cbe4ac6425ed74383bf065544366dfda2a82"}
Sep 18 22:32:08.222129 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:08.222090    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v" event={"ID":"8cf0ab99-2644-4647-a591-11789541c7f3","Type":"ContainerStarted","Data":"c09d9a2ecf848b068f33b003590c222324c7c1b5150cf765328d4bdbcc9eafaa"}
Sep 18 22:32:08.242620 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:08.242560    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v" podStartSLOduration=1.242545832 podStartE2EDuration="1.242545832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:32:08.239902155 +0000 UTC m=+912.600419014" watchObservedRunningTime="2026-09-18 22:32:08.242545832 +0000 UTC m=+912.603062688"
Sep 18 22:32:09.229132 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:09.229093    2583 generic.go:356] "Generic (PLEG): container finished" podID="8cf0ab99-2644-4647-a591-11789541c7f3" containerID="bbb87f6ee2c66c92c8bd04114947cbe4ac6425ed74383bf065544366dfda2a82" exitCode=0
Sep 18 22:32:09.229581 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:09.229198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v" event={"ID":"8cf0ab99-2644-4647-a591-11789541c7f3","Type":"ContainerDied","Data":"bbb87f6ee2c66c92c8bd04114947cbe4ac6425ed74383bf065544366dfda2a82"}
Sep 18 22:32:10.366028 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:10.366003    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:10.449012 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:10.448970    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf0ab99-2644-4647-a591-11789541c7f3-maas-api-service-ca\") pod \"8cf0ab99-2644-4647-a591-11789541c7f3\" (UID: \"8cf0ab99-2644-4647-a591-11789541c7f3\") "
Sep 18 22:32:10.449252 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:10.449229    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf0ab99-2644-4647-a591-11789541c7f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8cf0ab99-2644-4647-a591-11789541c7f3" (UID: "8cf0ab99-2644-4647-a591-11789541c7f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:32:10.449413 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:10.449370    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf0ab99-2644-4647-a591-11789541c7f3-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:32:11.243009 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:11.242977    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"
Sep 18 22:32:11.243185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:11.243018    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v" event={"ID":"8cf0ab99-2644-4647-a591-11789541c7f3","Type":"ContainerDied","Data":"c09d9a2ecf848b068f33b003590c222324c7c1b5150cf765328d4bdbcc9eafaa"}
Sep 18 22:32:11.243185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:11.243064    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c09d9a2ecf848b068f33b003590c222324c7c1b5150cf765328d4bdbcc9eafaa"
Sep 18 22:32:17.820232 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:17.820196    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"]
Sep 18 22:32:17.820708 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:17.820468    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw" podUID="b82f9253-cf6f-40f4-a639-bb89d6169b14" containerName="payload-processing" containerID="cri-o://42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa" gracePeriod=30
Sep 18 22:32:17.826465 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:17.826429    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"]
Sep 18 22:32:17.826916 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:17.826879    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f" podUID="629b85e4-bb48-4966-9a47-186df168996f" containerName="payload-pre-processing" containerID="cri-o://3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4" gracePeriod=30
Sep 18 22:32:17.987919 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:17.987882    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"]
Sep 18 22:32:17.991159 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:17.991083    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-49880939-pxw4v"]
Sep 18 22:32:18.088354 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.088326    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:32:18.137846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.137822    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:32:18.233767 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.233712    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcnb4\" (UniqueName: \"kubernetes.io/projected/629b85e4-bb48-4966-9a47-186df168996f-kube-api-access-lcnb4\") pod \"629b85e4-bb48-4966-9a47-186df168996f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") "
Sep 18 22:32:18.233988 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.233784    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629b85e4-bb48-4966-9a47-186df168996f-plugins-config-volume\") pod \"629b85e4-bb48-4966-9a47-186df168996f\" (UID: \"629b85e4-bb48-4966-9a47-186df168996f\") "
Sep 18 22:32:18.233988 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.233810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpncx\" (UniqueName: \"kubernetes.io/projected/b82f9253-cf6f-40f4-a639-bb89d6169b14-kube-api-access-cpncx\") pod \"b82f9253-cf6f-40f4-a639-bb89d6169b14\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") "
Sep 18 22:32:18.233988 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.233842    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82f9253-cf6f-40f4-a639-bb89d6169b14-plugins-config-volume\") pod \"b82f9253-cf6f-40f4-a639-bb89d6169b14\" (UID: \"b82f9253-cf6f-40f4-a639-bb89d6169b14\") "
Sep 18 22:32:18.234317 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.234293    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/629b85e4-bb48-4966-9a47-186df168996f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "629b85e4-bb48-4966-9a47-186df168996f" (UID: "629b85e4-bb48-4966-9a47-186df168996f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:32:18.234435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.234338    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b82f9253-cf6f-40f4-a639-bb89d6169b14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b82f9253-cf6f-40f4-a639-bb89d6169b14" (UID: "b82f9253-cf6f-40f4-a639-bb89d6169b14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:32:18.236108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.236070    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82f9253-cf6f-40f4-a639-bb89d6169b14-kube-api-access-cpncx" (OuterVolumeSpecName: "kube-api-access-cpncx") pod "b82f9253-cf6f-40f4-a639-bb89d6169b14" (UID: "b82f9253-cf6f-40f4-a639-bb89d6169b14"). InnerVolumeSpecName "kube-api-access-cpncx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:32:18.236212 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.236121    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629b85e4-bb48-4966-9a47-186df168996f-kube-api-access-lcnb4" (OuterVolumeSpecName: "kube-api-access-lcnb4") pod "629b85e4-bb48-4966-9a47-186df168996f" (UID: "629b85e4-bb48-4966-9a47-186df168996f"). InnerVolumeSpecName "kube-api-access-lcnb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:32:18.284273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.284237    2583 generic.go:356] "Generic (PLEG): container finished" podID="b82f9253-cf6f-40f4-a639-bb89d6169b14" containerID="42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa" exitCode=0
Sep 18 22:32:18.284457 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.284292    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"
Sep 18 22:32:18.284457 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.284326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw" event={"ID":"b82f9253-cf6f-40f4-a639-bb89d6169b14","Type":"ContainerDied","Data":"42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa"}
Sep 18 22:32:18.284457 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.284441    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw" event={"ID":"b82f9253-cf6f-40f4-a639-bb89d6169b14","Type":"ContainerDied","Data":"ccf2f3ec27da61478a9d0aba9b00f606d98297b22e36fe2e64cdfb6c881963fb"}
Sep 18 22:32:18.284615 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.284469    2583 scope.go:117] "RemoveContainer" containerID="42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa"
Sep 18 22:32:18.285953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.285917    2583 generic.go:356] "Generic (PLEG): container finished" podID="629b85e4-bb48-4966-9a47-186df168996f" containerID="3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4" exitCode=0
Sep 18 22:32:18.286069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.285957    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"
Sep 18 22:32:18.286069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.285963    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f" event={"ID":"629b85e4-bb48-4966-9a47-186df168996f","Type":"ContainerDied","Data":"3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4"}
Sep 18 22:32:18.286069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.286013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f" event={"ID":"629b85e4-bb48-4966-9a47-186df168996f","Type":"ContainerDied","Data":"1179fb363de4283252c7ce514d566fae3ebf00a00b699efc3b2ba20e54ab987e"}
Sep 18 22:32:18.295057 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.295038    2583 scope.go:117] "RemoveContainer" containerID="42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa"
Sep 18 22:32:18.295327 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:32:18.295304    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa\": container with ID starting with 42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa not found: ID does not exist" containerID="42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa"
Sep 18 22:32:18.295374 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.295337    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa"} err="failed to get container status \"42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa\": rpc error: code = NotFound desc = could not find container \"42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa\": container with ID starting with 42df837759744dae77b28372484de18dd77c34ffe130486f5ee6b362b5ec22fa not found: ID does not exist"
Sep 18 22:32:18.295374 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.295354    2583 scope.go:117] "RemoveContainer" containerID="3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4"
Sep 18 22:32:18.304158 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.304142    2583 scope.go:117] "RemoveContainer" containerID="3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4"
Sep 18 22:32:18.304444 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:32:18.304422    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4\": container with ID starting with 3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4 not found: ID does not exist" containerID="3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4"
Sep 18 22:32:18.304525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.304451    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4"} err="failed to get container status \"3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4\": rpc error: code = NotFound desc = could not find container \"3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4\": container with ID starting with 3c26ab0eeb931bab29eb759b9dea5f1bd5c06b44fd5d96bf2b362bfd3c569fd4 not found: ID does not exist"
Sep 18 22:32:18.314798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.312642    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"]
Sep 18 22:32:18.319042 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.319017    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49880939-bb6965b8b-mr29f"]
Sep 18 22:32:18.329599 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.329528    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"]
Sep 18 22:32:18.332709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.332685    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49880939-7587875649-nffhw"]
Sep 18 22:32:18.335521 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.335499    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82f9253-cf6f-40f4-a639-bb89d6169b14-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:32:18.335619 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.335523    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcnb4\" (UniqueName: \"kubernetes.io/projected/629b85e4-bb48-4966-9a47-186df168996f-kube-api-access-lcnb4\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:32:18.335619 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.335539    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629b85e4-bb48-4966-9a47-186df168996f-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:32:18.335619 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:18.335553    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpncx\" (UniqueName: \"kubernetes.io/projected/b82f9253-cf6f-40f4-a639-bb89d6169b14-kube-api-access-cpncx\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:32:19.685824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:19.685789    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="629b85e4-bb48-4966-9a47-186df168996f" path="/var/lib/kubelet/pods/629b85e4-bb48-4966-9a47-186df168996f/volumes"
Sep 18 22:32:19.686364 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:19.686323    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf0ab99-2644-4647-a591-11789541c7f3" path="/var/lib/kubelet/pods/8cf0ab99-2644-4647-a591-11789541c7f3/volumes"
Sep 18 22:32:19.686700 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:19.686685    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82f9253-cf6f-40f4-a639-bb89d6169b14" path="/var/lib/kubelet/pods/b82f9253-cf6f-40f4-a639-bb89d6169b14/volumes"
Sep 18 22:32:28.987526 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.987492    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"]
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988078    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf0ab99-2644-4647-a591-11789541c7f3" containerName="revoke-keys"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988094    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf0ab99-2644-4647-a591-11789541c7f3" containerName="revoke-keys"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988119    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b82f9253-cf6f-40f4-a639-bb89d6169b14" containerName="payload-processing"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988128    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b82f9253-cf6f-40f4-a639-bb89d6169b14" containerName="payload-processing"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988145    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="629b85e4-bb48-4966-9a47-186df168996f" containerName="payload-pre-processing"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988155    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="629b85e4-bb48-4966-9a47-186df168996f" containerName="payload-pre-processing"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988274    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="629b85e4-bb48-4966-9a47-186df168996f" containerName="payload-pre-processing"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988285    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b82f9253-cf6f-40f4-a639-bb89d6169b14" containerName="payload-processing"
Sep 18 22:32:28.989515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.988293    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf0ab99-2644-4647-a591-11789541c7f3" containerName="revoke-keys"
Sep 18 22:32:28.991620 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.991602    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:28.994435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:28.994411    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-292146-serving-cert\""
Sep 18 22:32:29.010532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.010501    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"]
Sep 18 22:32:29.132938 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.132896    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b52ce1-31c7-4790-9bff-35bdab313208-openshift-service-ca\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.133107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.133015    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5khfl\" (UniqueName: \"kubernetes.io/projected/58b52ce1-31c7-4790-9bff-35bdab313208-kube-api-access-5khfl\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.133107 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.133061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-metrics-certs\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.133183 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.133162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-maas-api-tls\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.233994 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.233964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-maas-api-tls\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.234165 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.234010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b52ce1-31c7-4790-9bff-35bdab313208-openshift-service-ca\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.234165 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.234054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5khfl\" (UniqueName: \"kubernetes.io/projected/58b52ce1-31c7-4790-9bff-35bdab313208-kube-api-access-5khfl\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.234165 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.234071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-metrics-certs\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.234696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.234668    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b52ce1-31c7-4790-9bff-35bdab313208-openshift-service-ca\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.236435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.236381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-maas-api-tls\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.236556 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.236436    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-metrics-certs\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.244990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.244932    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5khfl\" (UniqueName: \"kubernetes.io/projected/58b52ce1-31c7-4790-9bff-35bdab313208-kube-api-access-5khfl\") pod \"maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.302778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.302745    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:29.579734 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.579705    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"]
Sep 18 22:32:29.581323 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:32:29.581290    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b52ce1_31c7_4790_9bff_35bdab313208.slice/crio-7faec2ca08c25ec9b7aba1bed75fc8f15d527c5a85ac8eb12a39af80911f5218 WatchSource:0}: Error finding container 7faec2ca08c25ec9b7aba1bed75fc8f15d527c5a85ac8eb12a39af80911f5218: Status 404 returned error can't find the container with id 7faec2ca08c25ec9b7aba1bed75fc8f15d527c5a85ac8eb12a39af80911f5218
Sep 18 22:32:29.582634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.582614    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:32:29.858771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.858682    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"]
Sep 18 22:32:29.863448 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.863430    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:29.865984 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.865966    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-292146\""
Sep 18 22:32:29.866427 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.866413    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-292146-dockercfg-plgbb\""
Sep 18 22:32:29.874713 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:29.874691    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"]
Sep 18 22:32:30.041120 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.041083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-plugins-config-volume\") pod \"payload-processing-e2e-explicit-292146-854b75bf69-rxscn\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") " pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.041615 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.041177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xvpt\" (UniqueName: \"kubernetes.io/projected/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-kube-api-access-2xvpt\") pod \"payload-processing-e2e-explicit-292146-854b75bf69-rxscn\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") " pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.142526 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.142499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xvpt\" (UniqueName: \"kubernetes.io/projected/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-kube-api-access-2xvpt\") pod \"payload-processing-e2e-explicit-292146-854b75bf69-rxscn\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") " pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.142701 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.142605    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-plugins-config-volume\") pod \"payload-processing-e2e-explicit-292146-854b75bf69-rxscn\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") " pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.143215 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.143195    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-plugins-config-volume\") pod \"payload-processing-e2e-explicit-292146-854b75bf69-rxscn\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") " pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.158742 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.158711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xvpt\" (UniqueName: \"kubernetes.io/projected/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-kube-api-access-2xvpt\") pod \"payload-processing-e2e-explicit-292146-854b75bf69-rxscn\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") " pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.175399 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.175365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:30.360888 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.360852    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz" event={"ID":"58b52ce1-31c7-4790-9bff-35bdab313208","Type":"ContainerStarted","Data":"f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347"}
Sep 18 22:32:30.361097 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.360940    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz" event={"ID":"58b52ce1-31c7-4790-9bff-35bdab313208","Type":"ContainerStarted","Data":"7faec2ca08c25ec9b7aba1bed75fc8f15d527c5a85ac8eb12a39af80911f5218"}
Sep 18 22:32:30.361097 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.360985    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:30.382709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.382659    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz" podStartSLOduration=2.037070066 podStartE2EDuration="2.38264403s" totalImagesPullingTime="345.573964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:32:30.381334748 +0000 UTC m=+934.741851607" watchObservedRunningTime="2026-09-18 22:32:30.38264403 +0000 UTC m=+934.743160890"
Sep 18 22:32:30.455973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:30.455945    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"]
Sep 18 22:32:30.458215 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:32:30.458192    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd5c650_66d8_41d2_b23c_30bed5fd3be4.slice/crio-2f54ee8f15d232dbb5312caa4b34f3b83f56d3c59332ccf3dd6350661a50f022 WatchSource:0}: Error finding container 2f54ee8f15d232dbb5312caa4b34f3b83f56d3c59332ccf3dd6350661a50f022: Status 404 returned error can't find the container with id 2f54ee8f15d232dbb5312caa4b34f3b83f56d3c59332ccf3dd6350661a50f022
Sep 18 22:32:31.368672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:31.368613    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn" event={"ID":"7dd5c650-66d8-41d2-b23c-30bed5fd3be4","Type":"ContainerStarted","Data":"ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8"}
Sep 18 22:32:31.369104 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:31.368694    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn" event={"ID":"7dd5c650-66d8-41d2-b23c-30bed5fd3be4","Type":"ContainerStarted","Data":"2f54ee8f15d232dbb5312caa4b34f3b83f56d3c59332ccf3dd6350661a50f022"}
Sep 18 22:32:31.369104 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:31.368884    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:32:36.373891 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:36.373857    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:32:36.395736 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:36.395685    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn" podStartSLOduration=7.395665225 podStartE2EDuration="7.395665225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:32:31.401596093 +0000 UTC m=+935.762112952" watchObservedRunningTime="2026-09-18 22:32:36.395665225 +0000 UTC m=+940.756182085"
Sep 18 22:32:42.377414 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:32:42.377361    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:34:04.265869 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.265827    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"]
Sep 18 22:34:04.269894 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.269873    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.272597 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.272574    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-939c7fd8-serving-cert\""
Sep 18 22:34:04.280295 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.280270    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"]
Sep 18 22:34:04.396591 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.396550    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9ade642-b62b-448c-848c-c206f982ec08-openshift-service-ca\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.396788 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.396621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgl6v\" (UniqueName: \"kubernetes.io/projected/d9ade642-b62b-448c-848c-c206f982ec08-kube-api-access-rgl6v\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.396788 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.396651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-metrics-certs\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.396788 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.396690    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-maas-api-tls\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.497912 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.497866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9ade642-b62b-448c-848c-c206f982ec08-openshift-service-ca\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.498123 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.497943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgl6v\" (UniqueName: \"kubernetes.io/projected/d9ade642-b62b-448c-848c-c206f982ec08-kube-api-access-rgl6v\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.498123 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.497971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-metrics-certs\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.498123 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.498020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-maas-api-tls\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.498672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.498651    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9ade642-b62b-448c-848c-c206f982ec08-openshift-service-ca\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.500576 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.500555    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-metrics-certs\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.500677 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.500603    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-maas-api-tls\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.506496 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.506471    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgl6v\" (UniqueName: \"kubernetes.io/projected/d9ade642-b62b-448c-848c-c206f982ec08-kube-api-access-rgl6v\") pod \"maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.582025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.581940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:04.875474 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.875451    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"]
Sep 18 22:34:04.877473 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:34:04.877446    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ade642_b62b_448c_848c_c206f982ec08.slice/crio-56aec43313416fc6a8deba00efdaf3dfd97f507a947c58501cf596e6fcfa2822 WatchSource:0}: Error finding container 56aec43313416fc6a8deba00efdaf3dfd97f507a947c58501cf596e6fcfa2822: Status 404 returned error can't find the container with id 56aec43313416fc6a8deba00efdaf3dfd97f507a947c58501cf596e6fcfa2822
Sep 18 22:34:04.903770 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:04.903728    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp" event={"ID":"d9ade642-b62b-448c-848c-c206f982ec08","Type":"ContainerStarted","Data":"56aec43313416fc6a8deba00efdaf3dfd97f507a947c58501cf596e6fcfa2822"}
Sep 18 22:34:05.093063 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.093025    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"]
Sep 18 22:34:05.097764 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.097746    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.100190 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.100166    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-939c7fd8-dockercfg-t8czq\""
Sep 18 22:34:05.100318 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.100243    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-939c7fd8\""
Sep 18 22:34:05.109094 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.109069    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"]
Sep 18 22:34:05.203215 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.203183    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds4rx\" (UniqueName: \"kubernetes.io/projected/eebeead7-2b4e-4e67-9030-da1ac7d1410c-kube-api-access-ds4rx\") pod \"payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") " pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.203380 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.203241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eebeead7-2b4e-4e67-9030-da1ac7d1410c-plugins-config-volume\") pod \"payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") " pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.304010 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.303977    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds4rx\" (UniqueName: \"kubernetes.io/projected/eebeead7-2b4e-4e67-9030-da1ac7d1410c-kube-api-access-ds4rx\") pod \"payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") " pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.304412 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.304040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eebeead7-2b4e-4e67-9030-da1ac7d1410c-plugins-config-volume\") pod \"payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") " pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.304806 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.304783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eebeead7-2b4e-4e67-9030-da1ac7d1410c-plugins-config-volume\") pod \"payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") " pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.315530 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.315500    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds4rx\" (UniqueName: \"kubernetes.io/projected/eebeead7-2b4e-4e67-9030-da1ac7d1410c-kube-api-access-ds4rx\") pod \"payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") " pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.409221 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.409180    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.687767 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.687739    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"]
Sep 18 22:34:05.688871 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:34:05.688840    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebeead7_2b4e_4e67_9030_da1ac7d1410c.slice/crio-0716780ee5a831d7d8b48988e5fab3ff66c85bf8cb02a4f8274617c7d2d1d972 WatchSource:0}: Error finding container 0716780ee5a831d7d8b48988e5fab3ff66c85bf8cb02a4f8274617c7d2d1d972: Status 404 returned error can't find the container with id 0716780ee5a831d7d8b48988e5fab3ff66c85bf8cb02a4f8274617c7d2d1d972
Sep 18 22:34:05.911886 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.911849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4" event={"ID":"eebeead7-2b4e-4e67-9030-da1ac7d1410c","Type":"ContainerStarted","Data":"f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70"}
Sep 18 22:34:05.912071 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.911913    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:05.912071 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.911938    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4" event={"ID":"eebeead7-2b4e-4e67-9030-da1ac7d1410c","Type":"ContainerStarted","Data":"0716780ee5a831d7d8b48988e5fab3ff66c85bf8cb02a4f8274617c7d2d1d972"}
Sep 18 22:34:05.913519 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.913496    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp" event={"ID":"d9ade642-b62b-448c-848c-c206f982ec08","Type":"ContainerStarted","Data":"1a54d52f456684ab59f17546ffdbefa4c4d51c1faaba64bf7ddb2648f0afaac3"}
Sep 18 22:34:05.913643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.913625    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:05.932002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.931950    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4" podStartSLOduration=0.931934903 podStartE2EDuration="931.934903ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:34:05.931243862 +0000 UTC m=+1030.291760719" watchObservedRunningTime="2026-09-18 22:34:05.931934903 +0000 UTC m=+1030.292451761"
Sep 18 22:34:05.953403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:05.953346    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp" podStartSLOduration=1.602622845 podStartE2EDuration="1.953331126s" totalImagesPullingTime="350.708281ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:34:05.951032675 +0000 UTC m=+1030.311549566" watchObservedRunningTime="2026-09-18 22:34:05.953331126 +0000 UTC m=+1030.313848044"
Sep 18 22:34:11.925050 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:11.925019    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:16.921944 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:16.921913    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:33.255042 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.255008    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"]
Sep 18 22:34:33.255555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.255231    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz" podUID="58b52ce1-31c7-4790-9bff-35bdab313208" containerName="maas-api" containerID="cri-o://f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347" gracePeriod=30
Sep 18 22:34:33.334637 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.334603    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"]
Sep 18 22:34:33.334834 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.334810    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn" podUID="7dd5c650-66d8-41d2-b23c-30bed5fd3be4" containerName="payload-processing" containerID="cri-o://ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8" gracePeriod=30
Sep 18 22:34:33.624943 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.624908    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:34:33.631304 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.631284    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:34:33.664629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664588    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5khfl\" (UniqueName: \"kubernetes.io/projected/58b52ce1-31c7-4790-9bff-35bdab313208-kube-api-access-5khfl\") pod \"58b52ce1-31c7-4790-9bff-35bdab313208\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") "
Sep 18 22:34:33.664629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664625    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b52ce1-31c7-4790-9bff-35bdab313208-openshift-service-ca\") pod \"58b52ce1-31c7-4790-9bff-35bdab313208\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") "
Sep 18 22:34:33.664865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-metrics-certs\") pod \"58b52ce1-31c7-4790-9bff-35bdab313208\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") "
Sep 18 22:34:33.664865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664860    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-plugins-config-volume\") pod \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") "
Sep 18 22:34:33.664991 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664899    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xvpt\" (UniqueName: \"kubernetes.io/projected/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-kube-api-access-2xvpt\") pod \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\" (UID: \"7dd5c650-66d8-41d2-b23c-30bed5fd3be4\") "
Sep 18 22:34:33.664991 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664964    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-maas-api-tls\") pod \"58b52ce1-31c7-4790-9bff-35bdab313208\" (UID: \"58b52ce1-31c7-4790-9bff-35bdab313208\") "
Sep 18 22:34:33.664991 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.664972    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b52ce1-31c7-4790-9bff-35bdab313208-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58b52ce1-31c7-4790-9bff-35bdab313208" (UID: "58b52ce1-31c7-4790-9bff-35bdab313208"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:34:33.665309 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.665276    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b52ce1-31c7-4790-9bff-35bdab313208-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:33.665642 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.665613    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7dd5c650-66d8-41d2-b23c-30bed5fd3be4" (UID: "7dd5c650-66d8-41d2-b23c-30bed5fd3be4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:34:33.667469 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.667430    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-kube-api-access-2xvpt" (OuterVolumeSpecName: "kube-api-access-2xvpt") pod "7dd5c650-66d8-41d2-b23c-30bed5fd3be4" (UID: "7dd5c650-66d8-41d2-b23c-30bed5fd3be4"). InnerVolumeSpecName "kube-api-access-2xvpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:34:33.667469 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.667443    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b52ce1-31c7-4790-9bff-35bdab313208-kube-api-access-5khfl" (OuterVolumeSpecName: "kube-api-access-5khfl") pod "58b52ce1-31c7-4790-9bff-35bdab313208" (UID: "58b52ce1-31c7-4790-9bff-35bdab313208"). InnerVolumeSpecName "kube-api-access-5khfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:34:33.667622 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.667532    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58b52ce1-31c7-4790-9bff-35bdab313208" (UID: "58b52ce1-31c7-4790-9bff-35bdab313208"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:34:33.668342 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.668318    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58b52ce1-31c7-4790-9bff-35bdab313208" (UID: "58b52ce1-31c7-4790-9bff-35bdab313208"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:34:33.766897 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.766865    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:33.766897 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.766894    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5khfl\" (UniqueName: \"kubernetes.io/projected/58b52ce1-31c7-4790-9bff-35bdab313208-kube-api-access-5khfl\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:33.766897 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.766905    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b52ce1-31c7-4790-9bff-35bdab313208-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:33.767155 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.766915    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:33.767155 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:33.766923    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xvpt\" (UniqueName: \"kubernetes.io/projected/7dd5c650-66d8-41d2-b23c-30bed5fd3be4-kube-api-access-2xvpt\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:34.090537 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.090496    2583 generic.go:356] "Generic (PLEG): container finished" podID="58b52ce1-31c7-4790-9bff-35bdab313208" containerID="f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347" exitCode=0
Sep 18 22:34:34.090720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.090553    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"
Sep 18 22:34:34.090720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.090581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz" event={"ID":"58b52ce1-31c7-4790-9bff-35bdab313208","Type":"ContainerDied","Data":"f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347"}
Sep 18 22:34:34.090720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.090639    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz" event={"ID":"58b52ce1-31c7-4790-9bff-35bdab313208","Type":"ContainerDied","Data":"7faec2ca08c25ec9b7aba1bed75fc8f15d527c5a85ac8eb12a39af80911f5218"}
Sep 18 22:34:34.090720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.090666    2583 scope.go:117] "RemoveContainer" containerID="f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347"
Sep 18 22:34:34.092534 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.092505    2583 generic.go:356] "Generic (PLEG): container finished" podID="7dd5c650-66d8-41d2-b23c-30bed5fd3be4" containerID="ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8" exitCode=0
Sep 18 22:34:34.092654 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.092541    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"
Sep 18 22:34:34.092654 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.092578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn" event={"ID":"7dd5c650-66d8-41d2-b23c-30bed5fd3be4","Type":"ContainerDied","Data":"ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8"}
Sep 18 22:34:34.092654 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.092622    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn" event={"ID":"7dd5c650-66d8-41d2-b23c-30bed5fd3be4","Type":"ContainerDied","Data":"2f54ee8f15d232dbb5312caa4b34f3b83f56d3c59332ccf3dd6350661a50f022"}
Sep 18 22:34:34.100794 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.100776    2583 scope.go:117] "RemoveContainer" containerID="f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347"
Sep 18 22:34:34.101066 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:34:34.101045    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347\": container with ID starting with f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347 not found: ID does not exist" containerID="f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347"
Sep 18 22:34:34.101172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.101073    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347"} err="failed to get container status \"f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347\": rpc error: code = NotFound desc = could not find container \"f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347\": container with ID starting with f25b25d73495b673647277630ac041f4f92c60c8ff1d738b5b037d451d1ac347 not found: ID does not exist"
Sep 18 22:34:34.101172 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.101096    2583 scope.go:117] "RemoveContainer" containerID="ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8"
Sep 18 22:34:34.109094 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.109078    2583 scope.go:117] "RemoveContainer" containerID="ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8"
Sep 18 22:34:34.109343 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:34:34.109326    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8\": container with ID starting with ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8 not found: ID does not exist" containerID="ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8"
Sep 18 22:34:34.109410 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.109348    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8"} err="failed to get container status \"ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8\": rpc error: code = NotFound desc = could not find container \"ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8\": container with ID starting with ba223fe88e799b36256ba6777284d4fef77e05d198d252ffc10b3a3ae0ad3de8 not found: ID does not exist"
Sep 18 22:34:34.118295 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.118274    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"]
Sep 18 22:34:34.123037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.123018    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-292146-7445b4cb4f-sfpjz"]
Sep 18 22:34:34.137151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.137128    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"]
Sep 18 22:34:34.150759 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:34.150733    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-292146-854b75bf69-rxscn"]
Sep 18 22:34:35.683439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:35.683405    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b52ce1-31c7-4790-9bff-35bdab313208" path="/var/lib/kubelet/pods/58b52ce1-31c7-4790-9bff-35bdab313208/volumes"
Sep 18 22:34:35.683977 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:35.683953    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dd5c650-66d8-41d2-b23c-30bed5fd3be4" path="/var/lib/kubelet/pods/7dd5c650-66d8-41d2-b23c-30bed5fd3be4/volumes"
Sep 18 22:34:49.015641 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.015562    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"]
Sep 18 22:34:49.016147 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.015805    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp" podUID="d9ade642-b62b-448c-848c-c206f982ec08" containerName="maas-api" containerID="cri-o://1a54d52f456684ab59f17546ffdbefa4c4d51c1faaba64bf7ddb2648f0afaac3" gracePeriod=30
Sep 18 22:34:49.078047 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.078014    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"]
Sep 18 22:34:49.078463 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.078432    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4" podUID="eebeead7-2b4e-4e67-9030-da1ac7d1410c" containerName="payload-processing" containerID="cri-o://f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70" gracePeriod=30
Sep 18 22:34:49.186274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.186231    2583 generic.go:356] "Generic (PLEG): container finished" podID="d9ade642-b62b-448c-848c-c206f982ec08" containerID="1a54d52f456684ab59f17546ffdbefa4c4d51c1faaba64bf7ddb2648f0afaac3" exitCode=0
Sep 18 22:34:49.186403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.186295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp" event={"ID":"d9ade642-b62b-448c-848c-c206f982ec08","Type":"ContainerDied","Data":"1a54d52f456684ab59f17546ffdbefa4c4d51c1faaba64bf7ddb2648f0afaac3"}
Sep 18 22:34:49.341196 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.341169    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:49.387131 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.387098    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:49.412134 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.412099    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9ade642-b62b-448c-848c-c206f982ec08-openshift-service-ca\") pod \"d9ade642-b62b-448c-848c-c206f982ec08\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") "
Sep 18 22:34:49.412308 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.412197    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-metrics-certs\") pod \"d9ade642-b62b-448c-848c-c206f982ec08\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") "
Sep 18 22:34:49.412308 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.412231    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgl6v\" (UniqueName: \"kubernetes.io/projected/d9ade642-b62b-448c-848c-c206f982ec08-kube-api-access-rgl6v\") pod \"d9ade642-b62b-448c-848c-c206f982ec08\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") "
Sep 18 22:34:49.412308 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.412264    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-maas-api-tls\") pod \"d9ade642-b62b-448c-848c-c206f982ec08\" (UID: \"d9ade642-b62b-448c-848c-c206f982ec08\") "
Sep 18 22:34:49.412566 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.412535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9ade642-b62b-448c-848c-c206f982ec08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9ade642-b62b-448c-848c-c206f982ec08" (UID: "d9ade642-b62b-448c-848c-c206f982ec08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:34:49.412669 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.412650    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9ade642-b62b-448c-848c-c206f982ec08-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:49.414674 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.414637    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9ade642-b62b-448c-848c-c206f982ec08-kube-api-access-rgl6v" (OuterVolumeSpecName: "kube-api-access-rgl6v") pod "d9ade642-b62b-448c-848c-c206f982ec08" (UID: "d9ade642-b62b-448c-848c-c206f982ec08"). InnerVolumeSpecName "kube-api-access-rgl6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:34:49.414793 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.414724    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9ade642-b62b-448c-848c-c206f982ec08" (UID: "d9ade642-b62b-448c-848c-c206f982ec08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:34:49.415168 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.415144    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9ade642-b62b-448c-848c-c206f982ec08" (UID: "d9ade642-b62b-448c-848c-c206f982ec08"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:34:49.513433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.513377    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds4rx\" (UniqueName: \"kubernetes.io/projected/eebeead7-2b4e-4e67-9030-da1ac7d1410c-kube-api-access-ds4rx\") pod \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") "
Sep 18 22:34:49.513433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.513438    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eebeead7-2b4e-4e67-9030-da1ac7d1410c-plugins-config-volume\") pod \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\" (UID: \"eebeead7-2b4e-4e67-9030-da1ac7d1410c\") "
Sep 18 22:34:49.513697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.513653    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:49.513697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.513666    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgl6v\" (UniqueName: \"kubernetes.io/projected/d9ade642-b62b-448c-848c-c206f982ec08-kube-api-access-rgl6v\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:49.513697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.513676    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9ade642-b62b-448c-848c-c206f982ec08-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:49.513861 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.513770    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebeead7-2b4e-4e67-9030-da1ac7d1410c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eebeead7-2b4e-4e67-9030-da1ac7d1410c" (UID: "eebeead7-2b4e-4e67-9030-da1ac7d1410c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:34:49.515507 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.515476    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebeead7-2b4e-4e67-9030-da1ac7d1410c-kube-api-access-ds4rx" (OuterVolumeSpecName: "kube-api-access-ds4rx") pod "eebeead7-2b4e-4e67-9030-da1ac7d1410c" (UID: "eebeead7-2b4e-4e67-9030-da1ac7d1410c"). InnerVolumeSpecName "kube-api-access-ds4rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:34:49.614574 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.614481    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds4rx\" (UniqueName: \"kubernetes.io/projected/eebeead7-2b4e-4e67-9030-da1ac7d1410c-kube-api-access-ds4rx\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:49.614574 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:49.614509    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eebeead7-2b4e-4e67-9030-da1ac7d1410c-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:34:50.194093 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.194060    2583 generic.go:356] "Generic (PLEG): container finished" podID="eebeead7-2b4e-4e67-9030-da1ac7d1410c" containerID="f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70" exitCode=0
Sep 18 22:34:50.194586 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.194113    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"
Sep 18 22:34:50.194586 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.194152    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4" event={"ID":"eebeead7-2b4e-4e67-9030-da1ac7d1410c","Type":"ContainerDied","Data":"f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70"}
Sep 18 22:34:50.194586 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.194215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4" event={"ID":"eebeead7-2b4e-4e67-9030-da1ac7d1410c","Type":"ContainerDied","Data":"0716780ee5a831d7d8b48988e5fab3ff66c85bf8cb02a4f8274617c7d2d1d972"}
Sep 18 22:34:50.194586 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.194247    2583 scope.go:117] "RemoveContainer" containerID="f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70"
Sep 18 22:34:50.195834 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.195817    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"
Sep 18 22:34:50.195952 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.195844    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp" event={"ID":"d9ade642-b62b-448c-848c-c206f982ec08","Type":"ContainerDied","Data":"56aec43313416fc6a8deba00efdaf3dfd97f507a947c58501cf596e6fcfa2822"}
Sep 18 22:34:50.203804 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.203780    2583 scope.go:117] "RemoveContainer" containerID="f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70"
Sep 18 22:34:50.204091 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:34:50.204070    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70\": container with ID starting with f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70 not found: ID does not exist" containerID="f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70"
Sep 18 22:34:50.204170 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.204100    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70"} err="failed to get container status \"f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70\": rpc error: code = NotFound desc = could not find container \"f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70\": container with ID starting with f3ff0e1181c1b5cf509fc6d39b06cb9b11ee99f67ea5785be04820dfcbf02c70 not found: ID does not exist"
Sep 18 22:34:50.204170 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.204123    2583 scope.go:117] "RemoveContainer" containerID="1a54d52f456684ab59f17546ffdbefa4c4d51c1faaba64bf7ddb2648f0afaac3"
Sep 18 22:34:50.215246 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.215222    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"]
Sep 18 22:34:50.217812 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.217792    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-939c7fd8-6c875d6cdf-n9pdp"]
Sep 18 22:34:50.225761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.225741    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"]
Sep 18 22:34:50.230532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:50.230511    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-939c7fd8-679b99c6bf-2vcb4"]
Sep 18 22:34:51.683439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:51.683375    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9ade642-b62b-448c-848c-c206f982ec08" path="/var/lib/kubelet/pods/d9ade642-b62b-448c-848c-c206f982ec08/volumes"
Sep 18 22:34:51.683817 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:34:51.683783    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eebeead7-2b4e-4e67-9030-da1ac7d1410c" path="/var/lib/kubelet/pods/eebeead7-2b4e-4e67-9030-da1ac7d1410c/volumes"
Sep 18 22:35:18.583963 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.583928    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"]
Sep 18 22:35:18.584365 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584351    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ade642-b62b-448c-848c-c206f982ec08" containerName="maas-api"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584366    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ade642-b62b-448c-848c-c206f982ec08" containerName="maas-api"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584398    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eebeead7-2b4e-4e67-9030-da1ac7d1410c" containerName="payload-processing"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584407    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebeead7-2b4e-4e67-9030-da1ac7d1410c" containerName="payload-processing"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584423    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b52ce1-31c7-4790-9bff-35bdab313208" containerName="maas-api"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584429    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b52ce1-31c7-4790-9bff-35bdab313208" containerName="maas-api"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584435    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd5c650-66d8-41d2-b23c-30bed5fd3be4" containerName="payload-processing"
Sep 18 22:35:18.584444 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584440    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd5c650-66d8-41d2-b23c-30bed5fd3be4" containerName="payload-processing"
Sep 18 22:35:18.584980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584500    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9ade642-b62b-448c-848c-c206f982ec08" containerName="maas-api"
Sep 18 22:35:18.584980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584510    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="eebeead7-2b4e-4e67-9030-da1ac7d1410c" containerName="payload-processing"
Sep 18 22:35:18.584980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584518    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dd5c650-66d8-41d2-b23c-30bed5fd3be4" containerName="payload-processing"
Sep 18 22:35:18.584980 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.584523    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b52ce1-31c7-4790-9bff-35bdab313208" containerName="maas-api"
Sep 18 22:35:18.587761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.587742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.600035 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.600013    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-8faaa1-serving-cert\""
Sep 18 22:35:18.614887 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.614856    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"]
Sep 18 22:35:18.687334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.687302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.687525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.687354    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rq65\" (UniqueName: \"kubernetes.io/projected/56e9fa0a-325e-4e37-a94c-c25726a26184-kube-api-access-6rq65\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.687525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.687413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56e9fa0a-325e-4e37-a94c-c25726a26184-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.687525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.687505    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.788485 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.788437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.788673 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.788492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.788673 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.788557    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rq65\" (UniqueName: \"kubernetes.io/projected/56e9fa0a-325e-4e37-a94c-c25726a26184-kube-api-access-6rq65\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.788673 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.788593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56e9fa0a-325e-4e37-a94c-c25726a26184-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.789181 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.789156    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56e9fa0a-325e-4e37-a94c-c25726a26184-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.790945 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.790920    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.791026 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.790966    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.802663 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.802640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rq65\" (UniqueName: \"kubernetes.io/projected/56e9fa0a-325e-4e37-a94c-c25726a26184-kube-api-access-6rq65\") pod \"maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:18.897654 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:18.897624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:19.176520 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.176495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"]
Sep 18 22:35:19.178330 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:35:19.178292    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e9fa0a_325e_4e37_a94c_c25726a26184.slice/crio-9f2be614130303cb611c78494a3182e4e86cd4b6b8232c425ccdbc2381faa848 WatchSource:0}: Error finding container 9f2be614130303cb611c78494a3182e4e86cd4b6b8232c425ccdbc2381faa848: Status 404 returned error can't find the container with id 9f2be614130303cb611c78494a3182e4e86cd4b6b8232c425ccdbc2381faa848
Sep 18 22:35:19.355594 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.355557    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"]
Sep 18 22:35:19.360578 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.360552    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.364401 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.363284    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-8faaa1-dockercfg-tcvcn\""
Sep 18 22:35:19.364401 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.363751    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-8faaa1\""
Sep 18 22:35:19.375655 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.375626    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"]
Sep 18 22:35:19.376131 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.376104    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4" event={"ID":"56e9fa0a-325e-4e37-a94c-c25726a26184","Type":"ContainerStarted","Data":"9f2be614130303cb611c78494a3182e4e86cd4b6b8232c425ccdbc2381faa848"}
Sep 18 22:35:19.380120 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.380094    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"]
Sep 18 22:35:19.384309 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.384290    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.393596 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.393570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1809341b-65c6-4c63-a63c-33ba1e3a63d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.393718 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.393631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2559\" (UniqueName: \"kubernetes.io/projected/1809341b-65c6-4c63-a63c-33ba1e3a63d8-kube-api-access-j2559\") pod \"payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.394968 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.394947    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"]
Sep 18 22:35:19.498523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.498417    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2559\" (UniqueName: \"kubernetes.io/projected/1809341b-65c6-4c63-a63c-33ba1e3a63d8-kube-api-access-j2559\") pod \"payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.498523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.498482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98535521-9eeb-4bfc-917f-185903823c43-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.498762 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.498562    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jxnz\" (UniqueName: \"kubernetes.io/projected/98535521-9eeb-4bfc-917f-185903823c43-kube-api-access-9jxnz\") pod \"payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.498762 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.498645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1809341b-65c6-4c63-a63c-33ba1e3a63d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.499237 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.499220    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1809341b-65c6-4c63-a63c-33ba1e3a63d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.507455 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.507431    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2559\" (UniqueName: \"kubernetes.io/projected/1809341b-65c6-4c63-a63c-33ba1e3a63d8-kube-api-access-j2559\") pod \"payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.599106 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.599079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jxnz\" (UniqueName: \"kubernetes.io/projected/98535521-9eeb-4bfc-917f-185903823c43-kube-api-access-9jxnz\") pod \"payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.599378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.599161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98535521-9eeb-4bfc-917f-185903823c43-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.599770 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.599753    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98535521-9eeb-4bfc-917f-185903823c43-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.609098 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.609076    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jxnz\" (UniqueName: \"kubernetes.io/projected/98535521-9eeb-4bfc-917f-185903823c43-kube-api-access-9jxnz\") pod \"payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.675834 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.675797    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:19.696151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.696122    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:19.952908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.952868    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"]
Sep 18 22:35:19.954313 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:35:19.954286    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1809341b_65c6_4c63_a63c_33ba1e3a63d8.slice/crio-1e03a1b22f48140c6d0e306ce09897307cb7a90852f81fafe583f2d8cc496715 WatchSource:0}: Error finding container 1e03a1b22f48140c6d0e306ce09897307cb7a90852f81fafe583f2d8cc496715: Status 404 returned error can't find the container with id 1e03a1b22f48140c6d0e306ce09897307cb7a90852f81fafe583f2d8cc496715
Sep 18 22:35:19.985734 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:19.985711    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"]
Sep 18 22:35:19.989702 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:35:19.989672    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98535521_9eeb_4bfc_917f_185903823c43.slice/crio-fd7dc4d554a0d1dad3d9f164739056e43da167eab727611a46e70d2ba7b5ceb7 WatchSource:0}: Error finding container fd7dc4d554a0d1dad3d9f164739056e43da167eab727611a46e70d2ba7b5ceb7: Status 404 returned error can't find the container with id fd7dc4d554a0d1dad3d9f164739056e43da167eab727611a46e70d2ba7b5ceb7
Sep 18 22:35:20.384143 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.384105    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2" event={"ID":"1809341b-65c6-4c63-a63c-33ba1e3a63d8","Type":"ContainerStarted","Data":"90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0"}
Sep 18 22:35:20.384408 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.384164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2" event={"ID":"1809341b-65c6-4c63-a63c-33ba1e3a63d8","Type":"ContainerStarted","Data":"1e03a1b22f48140c6d0e306ce09897307cb7a90852f81fafe583f2d8cc496715"}
Sep 18 22:35:20.384408 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.384301    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:20.386048 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.386026    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx" event={"ID":"98535521-9eeb-4bfc-917f-185903823c43","Type":"ContainerStarted","Data":"a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5"}
Sep 18 22:35:20.386161 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.386065    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx" event={"ID":"98535521-9eeb-4bfc-917f-185903823c43","Type":"ContainerStarted","Data":"fd7dc4d554a0d1dad3d9f164739056e43da167eab727611a46e70d2ba7b5ceb7"}
Sep 18 22:35:20.386161 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.386095    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:20.387749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.387726    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4" event={"ID":"56e9fa0a-325e-4e37-a94c-c25726a26184","Type":"ContainerStarted","Data":"7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1"}
Sep 18 22:35:20.387849 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.387792    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:20.406656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.406607    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2" podStartSLOduration=1.406590403 podStartE2EDuration="1.406590403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:35:20.402954651 +0000 UTC m=+1104.763471525" watchObservedRunningTime="2026-09-18 22:35:20.406590403 +0000 UTC m=+1104.767107259"
Sep 18 22:35:20.424618 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.424566    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx" podStartSLOduration=1.424547893 podStartE2EDuration="1.424547893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:35:20.422978117 +0000 UTC m=+1104.783494998" watchObservedRunningTime="2026-09-18 22:35:20.424547893 +0000 UTC m=+1104.785064741"
Sep 18 22:35:20.447331 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:20.447275    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4" podStartSLOduration=2.092538832 podStartE2EDuration="2.447255478s" totalImagesPullingTime="354.716646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:35:20.444533905 +0000 UTC m=+1104.805050769" watchObservedRunningTime="2026-09-18 22:35:20.447255478 +0000 UTC m=+1104.807772336"
Sep 18 22:35:26.401160 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:26.401131    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:35:31.397969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:31.397939    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:35:31.398407 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:31.398015    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:35:54.107402 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.107353    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"]
Sep 18 22:35:54.111504 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.111476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.113973 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.113946    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-20531f-serving-cert\""
Sep 18 22:35:54.116761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.116683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.116958 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.116939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.117119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.117104    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91be9c9e-b867-4e96-8b49-4d70309d0180-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.117288 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.117271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb9p8\" (UniqueName: \"kubernetes.io/projected/91be9c9e-b867-4e96-8b49-4d70309d0180-kube-api-access-bb9p8\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.120552 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.120527    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"]
Sep 18 22:35:54.218067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.218031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.218267 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.218081    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.218267 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.218146    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91be9c9e-b867-4e96-8b49-4d70309d0180-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.218267 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:35:54.218167    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-serving-cert: secret "maas-api-e2e-shared-b-w2-20531f-serving-cert" not found
Sep 18 22:35:54.218267 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.218227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9p8\" (UniqueName: \"kubernetes.io/projected/91be9c9e-b867-4e96-8b49-4d70309d0180-kube-api-access-bb9p8\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.218267 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:35:54.218246    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls podName:91be9c9e-b867-4e96-8b49-4d70309d0180 nodeName:}" failed. No retries permitted until 2026-09-18 22:35:54.718223109 +0000 UTC m=+1139.078739958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls") pod "maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" (UID: "91be9c9e-b867-4e96-8b49-4d70309d0180") : secret "maas-api-e2e-shared-b-w2-20531f-serving-cert" not found
Sep 18 22:35:54.219274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.219243    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91be9c9e-b867-4e96-8b49-4d70309d0180-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.221555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.221518    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.226761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.226741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9p8\" (UniqueName: \"kubernetes.io/projected/91be9c9e-b867-4e96-8b49-4d70309d0180-kube-api-access-bb9p8\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.708110 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.708070    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"]
Sep 18 22:35:54.711970 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.711951    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:54.715567 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.715551    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-20531f\""
Sep 18 22:35:54.715643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.715584    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-20531f-dockercfg-fr2xn\""
Sep 18 22:35:54.722780 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.722745    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krqwc\" (UniqueName: \"kubernetes.io/projected/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-kube-api-access-krqwc\") pod \"payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:54.722920 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.722882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.723033 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.723011    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:54.725766 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.725739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:54.726771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.726751    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"]
Sep 18 22:35:54.824154 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.824117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:54.824346 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.824220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krqwc\" (UniqueName: \"kubernetes.io/projected/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-kube-api-access-krqwc\") pod \"payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:54.824990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.824965    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:54.836067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:54.836039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krqwc\" (UniqueName: \"kubernetes.io/projected/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-kube-api-access-krqwc\") pod \"payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:55.023096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.023003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:55.024105 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.024071    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:55.300666 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.300630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"]
Sep 18 22:35:55.301056 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:35:55.300969    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf52bda_1bf8_4cf2_aa67_4339dc817f20.slice/crio-7aa45f4e68e84057951059e01674cde0e0e3d0116eca4e72a82d43c4d996ae6c WatchSource:0}: Error finding container 7aa45f4e68e84057951059e01674cde0e0e3d0116eca4e72a82d43c4d996ae6c: Status 404 returned error can't find the container with id 7aa45f4e68e84057951059e01674cde0e0e3d0116eca4e72a82d43c4d996ae6c
Sep 18 22:35:55.311130 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.311099    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"]
Sep 18 22:35:55.317919 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:35:55.317893    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91be9c9e_b867_4e96_8b49_4d70309d0180.slice/crio-2e3706c091da8f6d3ef39e268f47f195edbbb5fb2cc8b6363bc190d36f96008a WatchSource:0}: Error finding container 2e3706c091da8f6d3ef39e268f47f195edbbb5fb2cc8b6363bc190d36f96008a: Status 404 returned error can't find the container with id 2e3706c091da8f6d3ef39e268f47f195edbbb5fb2cc8b6363bc190d36f96008a
Sep 18 22:35:55.593112 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.593015    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" event={"ID":"91be9c9e-b867-4e96-8b49-4d70309d0180","Type":"ContainerStarted","Data":"2e3706c091da8f6d3ef39e268f47f195edbbb5fb2cc8b6363bc190d36f96008a"}
Sep 18 22:35:55.594842 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.594817    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq" event={"ID":"3cf52bda-1bf8-4cf2-aa67-4339dc817f20","Type":"ContainerStarted","Data":"f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801"}
Sep 18 22:35:55.594999 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.594859    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq" event={"ID":"3cf52bda-1bf8-4cf2-aa67-4339dc817f20","Type":"ContainerStarted","Data":"7aa45f4e68e84057951059e01674cde0e0e3d0116eca4e72a82d43c4d996ae6c"}
Sep 18 22:35:55.594999 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.594935    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:35:55.615802 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:55.615750    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq" podStartSLOduration=1.615732851 podStartE2EDuration="1.615732851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:35:55.614684512 +0000 UTC m=+1139.975201374" watchObservedRunningTime="2026-09-18 22:35:55.615732851 +0000 UTC m=+1139.976249710"
Sep 18 22:35:56.603559 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:56.603521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" event={"ID":"91be9c9e-b867-4e96-8b49-4d70309d0180","Type":"ContainerStarted","Data":"02afc6f0c3e98b7413d6d21bbcde6023c959f8007fea7a7ecd395996a0ae6356"}
Sep 18 22:35:56.603974 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:56.603681    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:35:56.621286 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:56.621235    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" podStartSLOduration=2.304954329 podStartE2EDuration="2.621218876s" totalImagesPullingTime="316.264547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:35:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:35:56.619983899 +0000 UTC m=+1140.980500757" watchObservedRunningTime="2026-09-18 22:35:56.621218876 +0000 UTC m=+1140.981735733"
Sep 18 22:35:57.715318 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:35:57.715289    2583 scope.go:117] "RemoveContainer" containerID="2f4604b8eddc11f35f521ddf2920f740ed33a068204ab4325fcf6cc0f98ab4be"
Sep 18 22:36:02.614355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:02.614324    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:36:06.605836 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:06.605806    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:36:10.965616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:10.965515    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"]
Sep 18 22:36:10.966081 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:10.965772    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" podUID="bbae3e8f-5a90-43cb-a953-c9adf8a65f53" containerName="maas-api" containerID="cri-o://01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316" gracePeriod=30
Sep 18 22:36:11.045593 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.045019    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"]
Sep 18 22:36:11.045593 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.045406    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n" podUID="a318e0b6-d78c-4460-9013-866bc2503d81" containerName="payload-processing" containerID="cri-o://e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca" gracePeriod=30
Sep 18 22:36:11.056176 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.056142    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"]
Sep 18 22:36:11.058943 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.058890    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn" podUID="4ae8c507-8d59-4d72-80b2-a3ea29328130" containerName="payload-pre-processing" containerID="cri-o://600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97" gracePeriod=30
Sep 18 22:36:11.339583 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.339548    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:36:11.377618 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.377596    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:36:11.382790 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.382769    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrx2b\" (UniqueName: \"kubernetes.io/projected/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-kube-api-access-wrx2b\") pod \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") "
Sep 18 22:36:11.382892 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.382851    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-openshift-service-ca\") pod \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") "
Sep 18 22:36:11.382892 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.382884    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-metrics-certs\") pod \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") "
Sep 18 22:36:11.383012 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.382911    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls\") pod \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\" (UID: \"bbae3e8f-5a90-43cb-a953-c9adf8a65f53\") "
Sep 18 22:36:11.383224 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.383196    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bbae3e8f-5a90-43cb-a953-c9adf8a65f53" (UID: "bbae3e8f-5a90-43cb-a953-c9adf8a65f53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:36:11.383653 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.383632    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:36:11.385054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.385020    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bbae3e8f-5a90-43cb-a953-c9adf8a65f53" (UID: "bbae3e8f-5a90-43cb-a953-c9adf8a65f53"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:36:11.385171 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.385087    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-kube-api-access-wrx2b" (OuterVolumeSpecName: "kube-api-access-wrx2b") pod "bbae3e8f-5a90-43cb-a953-c9adf8a65f53" (UID: "bbae3e8f-5a90-43cb-a953-c9adf8a65f53"). InnerVolumeSpecName "kube-api-access-wrx2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:11.385327 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.385300    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bbae3e8f-5a90-43cb-a953-c9adf8a65f53" (UID: "bbae3e8f-5a90-43cb-a953-c9adf8a65f53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:36:11.484497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484412    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a318e0b6-d78c-4460-9013-866bc2503d81-plugins-config-volume\") pod \"a318e0b6-d78c-4460-9013-866bc2503d81\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") "
Sep 18 22:36:11.484497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484474    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx9xl\" (UniqueName: \"kubernetes.io/projected/a318e0b6-d78c-4460-9013-866bc2503d81-kube-api-access-jx9xl\") pod \"a318e0b6-d78c-4460-9013-866bc2503d81\" (UID: \"a318e0b6-d78c-4460-9013-866bc2503d81\") "
Sep 18 22:36:11.484682 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ae8c507-8d59-4d72-80b2-a3ea29328130-plugins-config-volume\") pod \"4ae8c507-8d59-4d72-80b2-a3ea29328130\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") "
Sep 18 22:36:11.484682 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484574    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tr56v\" (UniqueName: \"kubernetes.io/projected/4ae8c507-8d59-4d72-80b2-a3ea29328130-kube-api-access-tr56v\") pod \"4ae8c507-8d59-4d72-80b2-a3ea29328130\" (UID: \"4ae8c507-8d59-4d72-80b2-a3ea29328130\") "
Sep 18 22:36:11.484870 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484851    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.484939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484880    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.484939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484894    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.484939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484903    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrx2b\" (UniqueName: \"kubernetes.io/projected/bbae3e8f-5a90-43cb-a953-c9adf8a65f53-kube-api-access-wrx2b\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.484939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484846    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a318e0b6-d78c-4460-9013-866bc2503d81-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a318e0b6-d78c-4460-9013-866bc2503d81" (UID: "a318e0b6-d78c-4460-9013-866bc2503d81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:36:11.484939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.484909    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ae8c507-8d59-4d72-80b2-a3ea29328130-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ae8c507-8d59-4d72-80b2-a3ea29328130" (UID: "4ae8c507-8d59-4d72-80b2-a3ea29328130"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:36:11.486794 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.486766    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ae8c507-8d59-4d72-80b2-a3ea29328130-kube-api-access-tr56v" (OuterVolumeSpecName: "kube-api-access-tr56v") pod "4ae8c507-8d59-4d72-80b2-a3ea29328130" (UID: "4ae8c507-8d59-4d72-80b2-a3ea29328130"). InnerVolumeSpecName "kube-api-access-tr56v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:11.486908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.486797    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a318e0b6-d78c-4460-9013-866bc2503d81-kube-api-access-jx9xl" (OuterVolumeSpecName: "kube-api-access-jx9xl") pod "a318e0b6-d78c-4460-9013-866bc2503d81" (UID: "a318e0b6-d78c-4460-9013-866bc2503d81"). InnerVolumeSpecName "kube-api-access-jx9xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:11.586192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.586155    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a318e0b6-d78c-4460-9013-866bc2503d81-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.586192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.586186    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx9xl\" (UniqueName: \"kubernetes.io/projected/a318e0b6-d78c-4460-9013-866bc2503d81-kube-api-access-jx9xl\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.586192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.586195    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ae8c507-8d59-4d72-80b2-a3ea29328130-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.586192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.586203    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr56v\" (UniqueName: \"kubernetes.io/projected/4ae8c507-8d59-4d72-80b2-a3ea29328130-kube-api-access-tr56v\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:11.699272 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.699246    2583 generic.go:356] "Generic (PLEG): container finished" podID="4ae8c507-8d59-4d72-80b2-a3ea29328130" containerID="600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97" exitCode=0
Sep 18 22:36:11.699403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.699293    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"
Sep 18 22:36:11.699403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.699324    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn" event={"ID":"4ae8c507-8d59-4d72-80b2-a3ea29328130","Type":"ContainerDied","Data":"600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97"}
Sep 18 22:36:11.699403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.699376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn" event={"ID":"4ae8c507-8d59-4d72-80b2-a3ea29328130","Type":"ContainerDied","Data":"439c1da1421bcc95176f9db8414d12472eddde290f1d2e43f0bc5c1debb358c3"}
Sep 18 22:36:11.699570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.699416    2583 scope.go:117] "RemoveContainer" containerID="600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97"
Sep 18 22:36:11.700756 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.700733    2583 generic.go:356] "Generic (PLEG): container finished" podID="a318e0b6-d78c-4460-9013-866bc2503d81" containerID="e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca" exitCode=0
Sep 18 22:36:11.700879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.700794    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"
Sep 18 22:36:11.700879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.700807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n" event={"ID":"a318e0b6-d78c-4460-9013-866bc2503d81","Type":"ContainerDied","Data":"e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca"}
Sep 18 22:36:11.700879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.700842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n" event={"ID":"a318e0b6-d78c-4460-9013-866bc2503d81","Type":"ContainerDied","Data":"9160607b01cbb09b48d21056467cc9be4d7aac46f72da03a2afc4ebeffbc4f17"}
Sep 18 22:36:11.702833 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.702747    2583 generic.go:356] "Generic (PLEG): container finished" podID="bbae3e8f-5a90-43cb-a953-c9adf8a65f53" containerID="01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316" exitCode=0
Sep 18 22:36:11.702833 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.702796    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"
Sep 18 22:36:11.702833 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.702804    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" event={"ID":"bbae3e8f-5a90-43cb-a953-c9adf8a65f53","Type":"ContainerDied","Data":"01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316"}
Sep 18 22:36:11.703059 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.702858    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk" event={"ID":"bbae3e8f-5a90-43cb-a953-c9adf8a65f53","Type":"ContainerDied","Data":"ee98d1421fc47218942dfb288d8a670eb557e3fff4bc6fd6f5045b59cc890e08"}
Sep 18 22:36:11.709895 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.709877    2583 scope.go:117] "RemoveContainer" containerID="600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97"
Sep 18 22:36:11.710155 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:11.710134    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97\": container with ID starting with 600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97 not found: ID does not exist" containerID="600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97"
Sep 18 22:36:11.710227 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.710166    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97"} err="failed to get container status \"600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97\": rpc error: code = NotFound desc = could not find container \"600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97\": container with ID starting with 600fd1705980f014967b518cdee604fcfe761b2b9fda64434d5b3c8746652e97 not found: ID does not exist"
Sep 18 22:36:11.710227 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.710188    2583 scope.go:117] "RemoveContainer" containerID="e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca"
Sep 18 22:36:11.718301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.718271    2583 scope.go:117] "RemoveContainer" containerID="e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca"
Sep 18 22:36:11.718577 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:11.718554    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca\": container with ID starting with e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca not found: ID does not exist" containerID="e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca"
Sep 18 22:36:11.718656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.718584    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca"} err="failed to get container status \"e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca\": rpc error: code = NotFound desc = could not find container \"e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca\": container with ID starting with e3a099a55d57c5f013c6f3359546a5e0969148e49c3ffd5cdd9d1ccdfb7d88ca not found: ID does not exist"
Sep 18 22:36:11.718656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.718599    2583 scope.go:117] "RemoveContainer" containerID="01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316"
Sep 18 22:36:11.726556 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.726536    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"]
Sep 18 22:36:11.727297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.727276    2583 scope.go:117] "RemoveContainer" containerID="01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316"
Sep 18 22:36:11.727667 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:11.727640    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316\": container with ID starting with 01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316 not found: ID does not exist" containerID="01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316"
Sep 18 22:36:11.727776 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.727664    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316"} err="failed to get container status \"01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316\": rpc error: code = NotFound desc = could not find container \"01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316\": container with ID starting with 01ed2ad2eb3f9d80276f7aa7c82fb714f64a8c6e83dcf575eb173b221dd34316 not found: ID does not exist"
Sep 18 22:36:11.731026 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.731006    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eac715-5c64f85b76-99qgk"]
Sep 18 22:36:11.741953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.741893    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"]
Sep 18 22:36:11.744265 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.744244    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eac715-66798f9f58-x9tcn"]
Sep 18 22:36:11.754717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.754694    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"]
Sep 18 22:36:11.756863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:11.756846    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-eac715-5fd5cffc5c-4sw2n"]
Sep 18 22:36:12.010007 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.009933    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7541-9e107d"]
Sep 18 22:36:12.010378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010364    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbae3e8f-5a90-43cb-a953-c9adf8a65f53" containerName="maas-api"
Sep 18 22:36:12.010459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010380    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbae3e8f-5a90-43cb-a953-c9adf8a65f53" containerName="maas-api"
Sep 18 22:36:12.010459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010410    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a318e0b6-d78c-4460-9013-866bc2503d81" containerName="payload-processing"
Sep 18 22:36:12.010459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010418    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a318e0b6-d78c-4460-9013-866bc2503d81" containerName="payload-processing"
Sep 18 22:36:12.010459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010434    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae8c507-8d59-4d72-80b2-a3ea29328130" containerName="payload-pre-processing"
Sep 18 22:36:12.010459 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010440    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae8c507-8d59-4d72-80b2-a3ea29328130" containerName="payload-pre-processing"
Sep 18 22:36:12.010653 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010512    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a318e0b6-d78c-4460-9013-866bc2503d81" containerName="payload-processing"
Sep 18 22:36:12.010653 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010522    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae8c507-8d59-4d72-80b2-a3ea29328130" containerName="payload-pre-processing"
Sep 18 22:36:12.010653 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.010529    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbae3e8f-5a90-43cb-a953-c9adf8a65f53" containerName="maas-api"
Sep 18 22:36:12.015033 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.015016    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:12.017502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.017480    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lgx4p\""
Sep 18 22:36:12.021603 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.021580    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7541-9e107d"]
Sep 18 22:36:12.091690 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.091657    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkl68\" (UniqueName: \"kubernetes.io/projected/11b5bae0-e7ea-4b8d-a244-80e46da9d164-kube-api-access-rkl68\") pod \"test-curl-7541-9e107d\" (UID: \"11b5bae0-e7ea-4b8d-a244-80e46da9d164\") " pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:12.193270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.193237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl68\" (UniqueName: \"kubernetes.io/projected/11b5bae0-e7ea-4b8d-a244-80e46da9d164-kube-api-access-rkl68\") pod \"test-curl-7541-9e107d\" (UID: \"11b5bae0-e7ea-4b8d-a244-80e46da9d164\") " pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:12.203513 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.203489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkl68\" (UniqueName: \"kubernetes.io/projected/11b5bae0-e7ea-4b8d-a244-80e46da9d164-kube-api-access-rkl68\") pod \"test-curl-7541-9e107d\" (UID: \"11b5bae0-e7ea-4b8d-a244-80e46da9d164\") " pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:12.326230 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.326153    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:12.601175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.601143    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7541-9e107d"]
Sep 18 22:36:12.602580 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:36:12.602554    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11b5bae0_e7ea_4b8d_a244_80e46da9d164.slice/crio-8c4461b9ef1994b978d143562b157b2a3f812df94f6c9e5fe000567e481b2f63 WatchSource:0}: Error finding container 8c4461b9ef1994b978d143562b157b2a3f812df94f6c9e5fe000567e481b2f63: Status 404 returned error can't find the container with id 8c4461b9ef1994b978d143562b157b2a3f812df94f6c9e5fe000567e481b2f63
Sep 18 22:36:12.712833 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:12.712801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7541-9e107d" event={"ID":"11b5bae0-e7ea-4b8d-a244-80e46da9d164","Type":"ContainerStarted","Data":"8c4461b9ef1994b978d143562b157b2a3f812df94f6c9e5fe000567e481b2f63"}
Sep 18 22:36:13.684278 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:13.684246    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ae8c507-8d59-4d72-80b2-a3ea29328130" path="/var/lib/kubelet/pods/4ae8c507-8d59-4d72-80b2-a3ea29328130/volumes"
Sep 18 22:36:13.684671 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:13.684656    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a318e0b6-d78c-4460-9013-866bc2503d81" path="/var/lib/kubelet/pods/a318e0b6-d78c-4460-9013-866bc2503d81/volumes"
Sep 18 22:36:13.685020 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:13.685008    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbae3e8f-5a90-43cb-a953-c9adf8a65f53" path="/var/lib/kubelet/pods/bbae3e8f-5a90-43cb-a953-c9adf8a65f53/volumes"
Sep 18 22:36:13.720323 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:13.720239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7541-9e107d" event={"ID":"11b5bae0-e7ea-4b8d-a244-80e46da9d164","Type":"ContainerStarted","Data":"1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b"}
Sep 18 22:36:13.750249 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:13.750194    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7541-9e107d" podStartSLOduration=1.910751124 podStartE2EDuration="2.750179777s" totalImagesPullingTime="839.428653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:36:13.749482608 +0000 UTC m=+1158.109999466" watchObservedRunningTime="2026-09-18 22:36:13.750179777 +0000 UTC m=+1158.110696634"
Sep 18 22:36:14.398797 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.398760    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7541-9e107d"]
Sep 18 22:36:14.408295 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.408210    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7541-9e107d"]
Sep 18 22:36:14.524299 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.524267    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7541-e2dc2e"]
Sep 18 22:36:14.524721 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.524707    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" containerName="test-curl-7541-9e107d"
Sep 18 22:36:14.524780 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.524723    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" containerName="test-curl-7541-9e107d"
Sep 18 22:36:14.524817 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.524790    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" containerName="test-curl-7541-9e107d"
Sep 18 22:36:14.529241 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.529222    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:14.533813 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.533783    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7541-e2dc2e"]
Sep 18 22:36:14.616409 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.616353    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h96r\" (UniqueName: \"kubernetes.io/projected/dd77fe42-1352-404f-a0aa-19edef93795e-kube-api-access-7h96r\") pod \"test-curl-7541-e2dc2e\" (UID: \"dd77fe42-1352-404f-a0aa-19edef93795e\") " pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:14.717602 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.717509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h96r\" (UniqueName: \"kubernetes.io/projected/dd77fe42-1352-404f-a0aa-19edef93795e-kube-api-access-7h96r\") pod \"test-curl-7541-e2dc2e\" (UID: \"dd77fe42-1352-404f-a0aa-19edef93795e\") " pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:14.725976 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.725949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h96r\" (UniqueName: \"kubernetes.io/projected/dd77fe42-1352-404f-a0aa-19edef93795e-kube-api-access-7h96r\") pod \"test-curl-7541-e2dc2e\" (UID: \"dd77fe42-1352-404f-a0aa-19edef93795e\") " pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:14.728935 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:14.728913    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7541-9e107d" container="test-curl-7541-9e107d"
Sep 18 22:36:14.839491 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:14.839465    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:15.121678 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:15.121646    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7541-e2dc2e"]
Sep 18 22:36:15.122424 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:36:15.122374    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd77fe42_1352_404f_a0aa_19edef93795e.slice/crio-0c78331d83d0256e0b840a92f28ee7d2b503d83b4797312497db1e4d0a3af4b8 WatchSource:0}: Error finding container 0c78331d83d0256e0b840a92f28ee7d2b503d83b4797312497db1e4d0a3af4b8: Status 404 returned error can't find the container with id 0c78331d83d0256e0b840a92f28ee7d2b503d83b4797312497db1e4d0a3af4b8
Sep 18 22:36:15.735917 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:15.735879    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7541-e2dc2e" event={"ID":"dd77fe42-1352-404f-a0aa-19edef93795e","Type":"ContainerStarted","Data":"a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1"}
Sep 18 22:36:15.736434 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:15.735928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7541-e2dc2e" event={"ID":"dd77fe42-1352-404f-a0aa-19edef93795e","Type":"ContainerStarted","Data":"0c78331d83d0256e0b840a92f28ee7d2b503d83b4797312497db1e4d0a3af4b8"}
Sep 18 22:36:15.736434 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:15.736102    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7541-9e107d" podUID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" containerName="test-curl-7541-9e107d" containerID="cri-o://1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b" gracePeriod=2
Sep 18 22:36:15.763637 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:15.763579    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7541-e2dc2e" podStartSLOduration=1.7635620680000001 podStartE2EDuration="1.763562068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:36:15.758086798 +0000 UTC m=+1160.118603656" watchObservedRunningTime="2026-09-18 22:36:15.763562068 +0000 UTC m=+1160.124078924"
Sep 18 22:36:16.417729 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.417681    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7541-e2dc2e"]
Sep 18 22:36:16.428710 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.428675    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7541-e2dc2e"]
Sep 18 22:36:16.549796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.549718    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7541-34cfe2"]
Sep 18 22:36:16.550123 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.550111    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd77fe42-1352-404f-a0aa-19edef93795e" containerName="test-curl-7541-e2dc2e"
Sep 18 22:36:16.550165 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.550125    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd77fe42-1352-404f-a0aa-19edef93795e" containerName="test-curl-7541-e2dc2e"
Sep 18 22:36:16.550200 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.550194    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd77fe42-1352-404f-a0aa-19edef93795e" containerName="test-curl-7541-e2dc2e"
Sep 18 22:36:16.553428 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.553406    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:16.566908 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.566881    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7541-34cfe2"]
Sep 18 22:36:16.638874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.638837    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rshvg\" (UniqueName: \"kubernetes.io/projected/0bcd2d64-bfa1-4a3c-a390-10430477779d-kube-api-access-rshvg\") pod \"test-curl-7541-34cfe2\" (UID: \"0bcd2d64-bfa1-4a3c-a390-10430477779d\") " pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:16.740255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.740225    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rshvg\" (UniqueName: \"kubernetes.io/projected/0bcd2d64-bfa1-4a3c-a390-10430477779d-kube-api-access-rshvg\") pod \"test-curl-7541-34cfe2\" (UID: \"0bcd2d64-bfa1-4a3c-a390-10430477779d\") " pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:16.741813 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:16.741787    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7541-e2dc2e" container="test-curl-7541-e2dc2e"
Sep 18 22:36:16.748151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.748129    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rshvg\" (UniqueName: \"kubernetes.io/projected/0bcd2d64-bfa1-4a3c-a390-10430477779d-kube-api-access-rshvg\") pod \"test-curl-7541-34cfe2\" (UID: \"0bcd2d64-bfa1-4a3c-a390-10430477779d\") " pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:16.866728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:16.866647    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:17.148028 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:17.147986    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7541-34cfe2"]
Sep 18 22:36:17.151884 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:36:17.151856    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bcd2d64_bfa1_4a3c_a390_10430477779d.slice/crio-8f3942269464a55bb932c0c70d1800c97e87a8d4bb561d660ccac68c68117718 WatchSource:0}: Error finding container 8f3942269464a55bb932c0c70d1800c97e87a8d4bb561d660ccac68c68117718: Status 404 returned error can't find the container with id 8f3942269464a55bb932c0c70d1800c97e87a8d4bb561d660ccac68c68117718
Sep 18 22:36:17.749398 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:17.749357    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7541-34cfe2" event={"ID":"0bcd2d64-bfa1-4a3c-a390-10430477779d","Type":"ContainerStarted","Data":"b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4"}
Sep 18 22:36:17.749770 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:17.749414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7541-34cfe2" event={"ID":"0bcd2d64-bfa1-4a3c-a390-10430477779d","Type":"ContainerStarted","Data":"8f3942269464a55bb932c0c70d1800c97e87a8d4bb561d660ccac68c68117718"}
Sep 18 22:36:17.749770 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:17.749550    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7541-e2dc2e" podUID="dd77fe42-1352-404f-a0aa-19edef93795e" containerName="test-curl-7541-e2dc2e" containerID="cri-o://a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1" gracePeriod=2
Sep 18 22:36:17.769943 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:17.769892    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7541-34cfe2" podStartSLOduration=1.769875436 podStartE2EDuration="1.769875436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:36:17.768096023 +0000 UTC m=+1162.128612882" watchObservedRunningTime="2026-09-18 22:36:17.769875436 +0000 UTC m=+1162.130392350"
Sep 18 22:36:17.918122 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:17.918096    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:18.052979 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.052883    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkl68\" (UniqueName: \"kubernetes.io/projected/11b5bae0-e7ea-4b8d-a244-80e46da9d164-kube-api-access-rkl68\") pod \"11b5bae0-e7ea-4b8d-a244-80e46da9d164\" (UID: \"11b5bae0-e7ea-4b8d-a244-80e46da9d164\") "
Sep 18 22:36:18.055096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.055063    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b5bae0-e7ea-4b8d-a244-80e46da9d164-kube-api-access-rkl68" (OuterVolumeSpecName: "kube-api-access-rkl68") pod "11b5bae0-e7ea-4b8d-a244-80e46da9d164" (UID: "11b5bae0-e7ea-4b8d-a244-80e46da9d164"). InnerVolumeSpecName "kube-api-access-rkl68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:18.153965 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.153933    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkl68\" (UniqueName: \"kubernetes.io/projected/11b5bae0-e7ea-4b8d-a244-80e46da9d164-kube-api-access-rkl68\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:18.386496 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.386463    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7541-34cfe2"]
Sep 18 22:36:18.391929 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.391902    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7541-34cfe2"]
Sep 18 22:36:18.756941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.756853    2583 generic.go:356] "Generic (PLEG): container finished" podID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" containerID="1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b" exitCode=137
Sep 18 22:36:18.756941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.756893    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:18.756941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.756939    2583 scope.go:117] "RemoveContainer" containerID="1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b"
Sep 18 22:36:18.757378 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:18.757240    2583 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7541-34cfe2" container="test-curl-7541-34cfe2"
Sep 18 22:36:18.766818 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.766797    2583 scope.go:117] "RemoveContainer" containerID="1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b"
Sep 18 22:36:18.767092 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:18.767074    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b\": container with ID starting with 1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b not found: ID does not exist" containerID="1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b"
Sep 18 22:36:18.767139 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:18.767100    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b"} err="failed to get container status \"1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b\": rpc error: code = NotFound desc = could not find container \"1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b\": container with ID starting with 1059d7968360f9401fff525e2cd427e4757a57681a73a7838f2b2aff3c75496b not found: ID does not exist"
Sep 18 22:36:18.768142 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:18.768118    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7541-9e107d\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" podUID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" pod="opendatahub/test-curl-7541-9e107d"
Sep 18 22:36:19.683098 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:19.683066    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b5bae0-e7ea-4b8d-a244-80e46da9d164" path="/var/lib/kubelet/pods/11b5bae0-e7ea-4b8d-a244-80e46da9d164/volumes"
Sep 18 22:36:19.763716 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:19.763678    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7541-34cfe2" podUID="0bcd2d64-bfa1-4a3c-a390-10430477779d" containerName="test-curl-7541-34cfe2" containerID="cri-o://b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4" gracePeriod=2
Sep 18 22:36:19.926031 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:19.926011    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:20.071719 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.071631    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h96r\" (UniqueName: \"kubernetes.io/projected/dd77fe42-1352-404f-a0aa-19edef93795e-kube-api-access-7h96r\") pod \"dd77fe42-1352-404f-a0aa-19edef93795e\" (UID: \"dd77fe42-1352-404f-a0aa-19edef93795e\") "
Sep 18 22:36:20.073752 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.073726    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd77fe42-1352-404f-a0aa-19edef93795e-kube-api-access-7h96r" (OuterVolumeSpecName: "kube-api-access-7h96r") pod "dd77fe42-1352-404f-a0aa-19edef93795e" (UID: "dd77fe42-1352-404f-a0aa-19edef93795e"). InnerVolumeSpecName "kube-api-access-7h96r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:20.172539 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.172505    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h96r\" (UniqueName: \"kubernetes.io/projected/dd77fe42-1352-404f-a0aa-19edef93795e-kube-api-access-7h96r\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:20.770724 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.770685    2583 generic.go:356] "Generic (PLEG): container finished" podID="dd77fe42-1352-404f-a0aa-19edef93795e" containerID="a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1" exitCode=137
Sep 18 22:36:20.771209 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.770747    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:20.771209 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.770813    2583 scope.go:117] "RemoveContainer" containerID="a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1"
Sep 18 22:36:20.780706 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.780689    2583 scope.go:117] "RemoveContainer" containerID="a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1"
Sep 18 22:36:20.780988 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:20.780970    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1\": container with ID starting with a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1 not found: ID does not exist" containerID="a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1"
Sep 18 22:36:20.781062 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:20.780993    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1"} err="failed to get container status \"a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1\": rpc error: code = NotFound desc = could not find container \"a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1\": container with ID starting with a7d8d7a61a6bde4b076a217d8be33dc2e107032e9584be94092a243e5cf4bac1 not found: ID does not exist"
Sep 18 22:36:20.781660 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:20.781639    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7541-e2dc2e\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" podUID="dd77fe42-1352-404f-a0aa-19edef93795e" pod="opendatahub/test-curl-7541-e2dc2e"
Sep 18 22:36:21.682640 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:21.682609    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd77fe42-1352-404f-a0aa-19edef93795e" path="/var/lib/kubelet/pods/dd77fe42-1352-404f-a0aa-19edef93795e/volumes"
Sep 18 22:36:21.917665 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:21.917636    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:21.988718 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:21.988637    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rshvg\" (UniqueName: \"kubernetes.io/projected/0bcd2d64-bfa1-4a3c-a390-10430477779d-kube-api-access-rshvg\") pod \"0bcd2d64-bfa1-4a3c-a390-10430477779d\" (UID: \"0bcd2d64-bfa1-4a3c-a390-10430477779d\") "
Sep 18 22:36:21.990697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:21.990656    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bcd2d64-bfa1-4a3c-a390-10430477779d-kube-api-access-rshvg" (OuterVolumeSpecName: "kube-api-access-rshvg") pod "0bcd2d64-bfa1-4a3c-a390-10430477779d" (UID: "0bcd2d64-bfa1-4a3c-a390-10430477779d"). InnerVolumeSpecName "kube-api-access-rshvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:22.089696 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:22.089666    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rshvg\" (UniqueName: \"kubernetes.io/projected/0bcd2d64-bfa1-4a3c-a390-10430477779d-kube-api-access-rshvg\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:22.786609 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:22.786571    2583 generic.go:356] "Generic (PLEG): container finished" podID="0bcd2d64-bfa1-4a3c-a390-10430477779d" containerID="b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4" exitCode=137
Sep 18 22:36:22.786787 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:22.786618    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:22.786787 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:22.786676    2583 scope.go:117] "RemoveContainer" containerID="b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4"
Sep 18 22:36:22.796218 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:22.796195    2583 scope.go:117] "RemoveContainer" containerID="b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4"
Sep 18 22:36:22.796623 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:22.796603    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4\": container with ID starting with b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4 not found: ID does not exist" containerID="b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4"
Sep 18 22:36:22.796700 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:22.796629    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4"} err="failed to get container status \"b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4\": rpc error: code = NotFound desc = could not find container \"b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4\": container with ID starting with b3bdb57c0223bf8e9b152fb8385fba453eeb87f83cb2a191b6e7ddd0e42835d4 not found: ID does not exist"
Sep 18 22:36:22.796756 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:22.796722    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7541-34cfe2\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" podUID="0bcd2d64-bfa1-4a3c-a390-10430477779d" pod="opendatahub/test-curl-7541-34cfe2"
Sep 18 22:36:23.682595 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:23.682553    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bcd2d64-bfa1-4a3c-a390-10430477779d" path="/var/lib/kubelet/pods/0bcd2d64-bfa1-4a3c-a390-10430477779d/volumes"
Sep 18 22:36:27.372882 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.372850    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"]
Sep 18 22:36:27.373335 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.373256    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bcd2d64-bfa1-4a3c-a390-10430477779d" containerName="test-curl-7541-34cfe2"
Sep 18 22:36:27.373335 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.373267    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bcd2d64-bfa1-4a3c-a390-10430477779d" containerName="test-curl-7541-34cfe2"
Sep 18 22:36:27.373431 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.373339    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bcd2d64-bfa1-4a3c-a390-10430477779d" containerName="test-curl-7541-34cfe2"
Sep 18 22:36:27.378160 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.378136    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.380561 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.380540    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-20e1128a-serving-cert\""
Sep 18 22:36:27.389558 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.389533    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"]
Sep 18 22:36:27.538577 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.538541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-metrics-certs\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.538732 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.538604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qph4h\" (UniqueName: \"kubernetes.io/projected/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-kube-api-access-qph4h\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.538732 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.538643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-openshift-service-ca\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.538732 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.538673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-maas-api-tls\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.640086 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.640051    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-metrics-certs\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.640306 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.640119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qph4h\" (UniqueName: \"kubernetes.io/projected/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-kube-api-access-qph4h\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.640306 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.640151    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-openshift-service-ca\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.640306 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.640182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-maas-api-tls\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.640895 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.640874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-openshift-service-ca\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.642749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.642722    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-metrics-certs\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.642749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.642734    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-maas-api-tls\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.648470 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.648445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qph4h\" (UniqueName: \"kubernetes.io/projected/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-kube-api-access-qph4h\") pod \"maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.690962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.690934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:27.977227 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:27.976702    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"]
Sep 18 22:36:27.978070 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:36:27.978042    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64261ca_eb23_4033_9fc6_49fd88a0c4d9.slice/crio-b81a2defbeb2dcfef8713684a8ab6cc863cee6ed0890d38dfa6e7c86d3c24a85 WatchSource:0}: Error finding container b81a2defbeb2dcfef8713684a8ab6cc863cee6ed0890d38dfa6e7c86d3c24a85: Status 404 returned error can't find the container with id b81a2defbeb2dcfef8713684a8ab6cc863cee6ed0890d38dfa6e7c86d3c24a85
Sep 18 22:36:28.087453 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.087420    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"]
Sep 18 22:36:28.092098 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.092081    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.095105 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.095080    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-20e1128a\""
Sep 18 22:36:28.095220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.095170    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-20e1128a-dockercfg-bvrqr\""
Sep 18 22:36:28.106516 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.106495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"]
Sep 18 22:36:28.122631 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.122594    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"]
Sep 18 22:36:28.126448 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.126428    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.145036 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.145003    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"]
Sep 18 22:36:28.247362 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.247277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe84c3a-eebc-4b18-9285-8774b351cb73-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.247362 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.247325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7525fdfb-e9e1-470b-9e09-3292855dde8e-plugins-config-volume\") pod \"payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") " pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.247362 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.247347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk95m\" (UniqueName: \"kubernetes.io/projected/7525fdfb-e9e1-470b-9e09-3292855dde8e-kube-api-access-hk95m\") pod \"payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") " pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.247612 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.247520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtt4g\" (UniqueName: \"kubernetes.io/projected/4fe84c3a-eebc-4b18-9285-8774b351cb73-kube-api-access-dtt4g\") pod \"payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.348791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.348753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtt4g\" (UniqueName: \"kubernetes.io/projected/4fe84c3a-eebc-4b18-9285-8774b351cb73-kube-api-access-dtt4g\") pod \"payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.348962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.348820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe84c3a-eebc-4b18-9285-8774b351cb73-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.348962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.348850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7525fdfb-e9e1-470b-9e09-3292855dde8e-plugins-config-volume\") pod \"payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") " pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.348962 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.348869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk95m\" (UniqueName: \"kubernetes.io/projected/7525fdfb-e9e1-470b-9e09-3292855dde8e-kube-api-access-hk95m\") pod \"payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") " pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.349563 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.349542    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe84c3a-eebc-4b18-9285-8774b351cb73-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.349627 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.349597    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7525fdfb-e9e1-470b-9e09-3292855dde8e-plugins-config-volume\") pod \"payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") " pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.360909 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.360882    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk95m\" (UniqueName: \"kubernetes.io/projected/7525fdfb-e9e1-470b-9e09-3292855dde8e-kube-api-access-hk95m\") pod \"payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") " pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.361054 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.360884    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtt4g\" (UniqueName: \"kubernetes.io/projected/4fe84c3a-eebc-4b18-9285-8774b351cb73-kube-api-access-dtt4g\") pod \"payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.403086 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.403057    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.442206 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.442183    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:28.702836 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.702809    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"]
Sep 18 22:36:28.705034 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:36:28.705004    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7525fdfb_e9e1_470b_9e09_3292855dde8e.slice/crio-3219e3f30c89f17c79b8d7a97f19c2e5a72b0f8489e2ef34a0eb321db7bc79ad WatchSource:0}: Error finding container 3219e3f30c89f17c79b8d7a97f19c2e5a72b0f8489e2ef34a0eb321db7bc79ad: Status 404 returned error can't find the container with id 3219e3f30c89f17c79b8d7a97f19c2e5a72b0f8489e2ef34a0eb321db7bc79ad
Sep 18 22:36:28.759927 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.759901    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"]
Sep 18 22:36:28.760965 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:36:28.760939    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe84c3a_eebc_4b18_9285_8774b351cb73.slice/crio-4f61df993e97ef82fa966c1102cbce5398a28df10ae1a87afa5b5c4d0c89978e WatchSource:0}: Error finding container 4f61df993e97ef82fa966c1102cbce5398a28df10ae1a87afa5b5c4d0c89978e: Status 404 returned error can't find the container with id 4f61df993e97ef82fa966c1102cbce5398a28df10ae1a87afa5b5c4d0c89978e
Sep 18 22:36:28.826776 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.826319    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp" event={"ID":"7525fdfb-e9e1-470b-9e09-3292855dde8e","Type":"ContainerStarted","Data":"057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb"}
Sep 18 22:36:28.827374 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.827341    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:28.827540 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.827464    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp" event={"ID":"7525fdfb-e9e1-470b-9e09-3292855dde8e","Type":"ContainerStarted","Data":"3219e3f30c89f17c79b8d7a97f19c2e5a72b0f8489e2ef34a0eb321db7bc79ad"}
Sep 18 22:36:28.828447 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.828425    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f" event={"ID":"f64261ca-eb23-4033-9fc6-49fd88a0c4d9","Type":"ContainerStarted","Data":"985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf"}
Sep 18 22:36:28.828547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.828467    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f" event={"ID":"f64261ca-eb23-4033-9fc6-49fd88a0c4d9","Type":"ContainerStarted","Data":"b81a2defbeb2dcfef8713684a8ab6cc863cee6ed0890d38dfa6e7c86d3c24a85"}
Sep 18 22:36:28.828547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.828492    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:28.829846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.829827    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd" event={"ID":"4fe84c3a-eebc-4b18-9285-8774b351cb73","Type":"ContainerStarted","Data":"4f61df993e97ef82fa966c1102cbce5398a28df10ae1a87afa5b5c4d0c89978e"}
Sep 18 22:36:28.887750 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.887691    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp" podStartSLOduration=0.887672909 podStartE2EDuration="887.672909ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:36:28.887416035 +0000 UTC m=+1173.247932888" watchObservedRunningTime="2026-09-18 22:36:28.887672909 +0000 UTC m=+1173.248189771"
Sep 18 22:36:28.932201 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:28.932140    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f" podStartSLOduration=1.529990226 podStartE2EDuration="1.932123126s" totalImagesPullingTime="402.1329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:36:28.929160259 +0000 UTC m=+1173.289677139" watchObservedRunningTime="2026-09-18 22:36:28.932123126 +0000 UTC m=+1173.292639982"
Sep 18 22:36:29.840735 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:29.840693    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd" event={"ID":"4fe84c3a-eebc-4b18-9285-8774b351cb73","Type":"ContainerStarted","Data":"4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94"}
Sep 18 22:36:29.841215 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:29.840780    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:29.862545 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:29.862492    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd" podStartSLOduration=1.86247298 podStartE2EDuration="1.86247298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:36:29.859882404 +0000 UTC m=+1174.220399288" watchObservedRunningTime="2026-09-18 22:36:29.86247298 +0000 UTC m=+1174.222989836"
Sep 18 22:36:34.848921 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:34.848827    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:36:35.817065 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:35.817026    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"]
Sep 18 22:36:35.817333 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:35.817286    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" podUID="118d4da0-e8ba-410e-9aff-57ae036f0657" containerName="maas-api" containerID="cri-o://8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d" gracePeriod=30
Sep 18 22:36:35.851950 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:35.851910    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"]
Sep 18 22:36:35.854644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:35.854618    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-44d735-29829510-2g6vf"]
Sep 18 22:36:36.075090 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.075027    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:36:36.222027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.221992    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls\") pod \"118d4da0-e8ba-410e-9aff-57ae036f0657\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") "
Sep 18 22:36:36.222217 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.222059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-metrics-certs\") pod \"118d4da0-e8ba-410e-9aff-57ae036f0657\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") "
Sep 18 22:36:36.222217 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.222137    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/118d4da0-e8ba-410e-9aff-57ae036f0657-openshift-service-ca\") pod \"118d4da0-e8ba-410e-9aff-57ae036f0657\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") "
Sep 18 22:36:36.222217 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.222170    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nng6\" (UniqueName: \"kubernetes.io/projected/118d4da0-e8ba-410e-9aff-57ae036f0657-kube-api-access-8nng6\") pod \"118d4da0-e8ba-410e-9aff-57ae036f0657\" (UID: \"118d4da0-e8ba-410e-9aff-57ae036f0657\") "
Sep 18 22:36:36.222608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.222571    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/118d4da0-e8ba-410e-9aff-57ae036f0657-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "118d4da0-e8ba-410e-9aff-57ae036f0657" (UID: "118d4da0-e8ba-410e-9aff-57ae036f0657"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:36:36.224294 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.224253    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "118d4da0-e8ba-410e-9aff-57ae036f0657" (UID: "118d4da0-e8ba-410e-9aff-57ae036f0657"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:36:36.224643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.224606    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/118d4da0-e8ba-410e-9aff-57ae036f0657-kube-api-access-8nng6" (OuterVolumeSpecName: "kube-api-access-8nng6") pod "118d4da0-e8ba-410e-9aff-57ae036f0657" (UID: "118d4da0-e8ba-410e-9aff-57ae036f0657"). InnerVolumeSpecName "kube-api-access-8nng6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:36:36.224737 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.224692    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "118d4da0-e8ba-410e-9aff-57ae036f0657" (UID: "118d4da0-e8ba-410e-9aff-57ae036f0657"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:36:36.322818 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.322782    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/118d4da0-e8ba-410e-9aff-57ae036f0657-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:36.322818 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.322811    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nng6\" (UniqueName: \"kubernetes.io/projected/118d4da0-e8ba-410e-9aff-57ae036f0657-kube-api-access-8nng6\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:36.322818 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.322821    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:36.323060 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.322830    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/118d4da0-e8ba-410e-9aff-57ae036f0657-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:36:36.887686 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.887653    2583 generic.go:356] "Generic (PLEG): container finished" podID="118d4da0-e8ba-410e-9aff-57ae036f0657" containerID="8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d" exitCode=0
Sep 18 22:36:36.888072 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.887704    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"
Sep 18 22:36:36.888072 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.887713    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" event={"ID":"118d4da0-e8ba-410e-9aff-57ae036f0657","Type":"ContainerDied","Data":"8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d"}
Sep 18 22:36:36.888072 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.887765    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd" event={"ID":"118d4da0-e8ba-410e-9aff-57ae036f0657","Type":"ContainerDied","Data":"8b90fdfa00ad6c261b0ad71f8ad517bc75dedcbd51c53737cd882b6347f11f40"}
Sep 18 22:36:36.888072 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.887795    2583 scope.go:117] "RemoveContainer" containerID="8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d"
Sep 18 22:36:36.898576 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.898550    2583 scope.go:117] "RemoveContainer" containerID="8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d"
Sep 18 22:36:36.898889 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:36.898867    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d\": container with ID starting with 8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d not found: ID does not exist" containerID="8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d"
Sep 18 22:36:36.898942 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.898901    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d"} err="failed to get container status \"8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d\": rpc error: code = NotFound desc = could not find container \"8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d\": container with ID starting with 8b533b3503d084d46c8a610d0b2e25ab66ea187e8f41e4b1fdf325cec7dbe73d not found: ID does not exist"
Sep 18 22:36:36.912401 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.912355    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"]
Sep 18 22:36:36.916341 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:36.916314    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-44d735-7898c584d4-mwxhd"]
Sep 18 22:36:36.938687 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:36:36.938653    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118d4da0_e8ba_410e_9aff_57ae036f0657.slice/crio-8b90fdfa00ad6c261b0ad71f8ad517bc75dedcbd51c53737cd882b6347f11f40\": RecentStats: unable to find data in memory cache]"
Sep 18 22:36:37.683972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:37.683938    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="118d4da0-e8ba-410e-9aff-57ae036f0657" path="/var/lib/kubelet/pods/118d4da0-e8ba-410e-9aff-57ae036f0657/volumes"
Sep 18 22:36:37.684257 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:37.684244    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1" path="/var/lib/kubelet/pods/ecd4d959-1ca7-42be-962d-6b3e4b1b7fc1/volumes"
Sep 18 22:36:40.848782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:40.848754    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:36:40.849189 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:40.848831    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:36:57.666320 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:57.666291    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:36:57.666809 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:57.666690    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:36:57.738209 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:36:57.738181    2583 scope.go:117] "RemoveContainer" containerID="b286ca71062857eb77dcfa94121ad5b00ed30d35aa99470cb618aad4d67d06d4"
Sep 18 22:37:22.123463 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.123424    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"]
Sep 18 22:37:22.123915 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.123833    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="118d4da0-e8ba-410e-9aff-57ae036f0657" containerName="maas-api"
Sep 18 22:37:22.123915 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.123843    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="118d4da0-e8ba-410e-9aff-57ae036f0657" containerName="maas-api"
Sep 18 22:37:22.123987 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.123921    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="118d4da0-e8ba-410e-9aff-57ae036f0657" containerName="maas-api"
Sep 18 22:37:22.127139 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.127122    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.129496 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.129476    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-a91c21-serving-cert\""
Sep 18 22:37:22.138273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.138241    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"]
Sep 18 22:37:22.235672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.235638    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.235940 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.235702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.235940 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.235841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbg7f\" (UniqueName: \"kubernetes.io/projected/db4fad39-1998-4a2c-85c8-545da2eafb75-kube-api-access-kbg7f\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.235940 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.235883    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4fad39-1998-4a2c-85c8-545da2eafb75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.337267 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.337227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.337477 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.337326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbg7f\" (UniqueName: \"kubernetes.io/projected/db4fad39-1998-4a2c-85c8-545da2eafb75-kube-api-access-kbg7f\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.337477 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.337357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4fad39-1998-4a2c-85c8-545da2eafb75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.337477 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.337380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.337477 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:37:22.337423    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-serving-cert: secret "maas-api-e2e-shared-b-w3-a91c21-serving-cert" not found
Sep 18 22:37:22.337642 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:37:22.337505    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls podName:db4fad39-1998-4a2c-85c8-545da2eafb75 nodeName:}" failed. No retries permitted until 2026-09-18 22:37:22.837485255 +0000 UTC m=+1227.198002105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls") pod "maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" (UID: "db4fad39-1998-4a2c-85c8-545da2eafb75") : secret "maas-api-e2e-shared-b-w3-a91c21-serving-cert" not found
Sep 18 22:37:22.338096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.338069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4fad39-1998-4a2c-85c8-545da2eafb75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.339733 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.339716    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.346577 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.346556    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbg7f\" (UniqueName: \"kubernetes.io/projected/db4fad39-1998-4a2c-85c8-545da2eafb75-kube-api-access-kbg7f\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.800423 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.800369    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"]
Sep 18 22:37:22.804274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.804259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:22.806835 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.806812    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-a91c21\""
Sep 18 22:37:22.820843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.820811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"]
Sep 18 22:37:22.840712 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.840683    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"]
Sep 18 22:37:22.843204 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.843182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.844134 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.844120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:22.845707 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.845682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:22.846871 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.846855    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-a91c21-dockercfg-9ktx6\""
Sep 18 22:37:22.862188 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.862164    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"]
Sep 18 22:37:22.944440 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.944407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:22.944440 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.944444    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tzsj\" (UniqueName: \"kubernetes.io/projected/f5342023-b122-43e8-aa1d-00c093d4ae5b-kube-api-access-9tzsj\") pod \"payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:22.944642 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.944488    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlp6d\" (UniqueName: \"kubernetes.io/projected/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-kube-api-access-dlp6d\") pod \"payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:22.944642 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:22.944620    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5342023-b122-43e8-aa1d-00c093d4ae5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:23.040178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.040144    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:23.045630 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.045592    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5342023-b122-43e8-aa1d-00c093d4ae5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:23.045771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.045662    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:23.045771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.045688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tzsj\" (UniqueName: \"kubernetes.io/projected/f5342023-b122-43e8-aa1d-00c093d4ae5b-kube-api-access-9tzsj\") pod \"payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:23.045771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.045739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlp6d\" (UniqueName: \"kubernetes.io/projected/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-kube-api-access-dlp6d\") pod \"payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:23.046313 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.046294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5342023-b122-43e8-aa1d-00c093d4ae5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:23.046471 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.046450    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:23.054057 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.054027    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tzsj\" (UniqueName: \"kubernetes.io/projected/f5342023-b122-43e8-aa1d-00c093d4ae5b-kube-api-access-9tzsj\") pod \"payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:23.054144 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.054087    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlp6d\" (UniqueName: \"kubernetes.io/projected/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-kube-api-access-dlp6d\") pod \"payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:23.115377 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.115330    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:23.166160 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.165945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:23.313484 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.313458    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"]
Sep 18 22:37:23.314933 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:37:23.314910    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4fad39_1998_4a2c_85c8_545da2eafb75.slice/crio-bd6b3af419b042cd1089a133b83cc781dc4f455614b8723ee42ece63a20f3613 WatchSource:0}: Error finding container bd6b3af419b042cd1089a133b83cc781dc4f455614b8723ee42ece63a20f3613: Status 404 returned error can't find the container with id bd6b3af419b042cd1089a133b83cc781dc4f455614b8723ee42ece63a20f3613
Sep 18 22:37:23.400993 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.400885    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"]
Sep 18 22:37:23.404673 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:37:23.404641    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5342023_b122_43e8_aa1d_00c093d4ae5b.slice/crio-bfe334fff2fe1a24613061a7acf3db3527688e12ca6db3bc2dee367dcbe0d99e WatchSource:0}: Error finding container bfe334fff2fe1a24613061a7acf3db3527688e12ca6db3bc2dee367dcbe0d99e: Status 404 returned error can't find the container with id bfe334fff2fe1a24613061a7acf3db3527688e12ca6db3bc2dee367dcbe0d99e
Sep 18 22:37:23.450996 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:23.450973    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"]
Sep 18 22:37:23.461532 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:37:23.461500    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e05e15_28de_404b_8e84_e9ad5fbebcd1.slice/crio-78e4a8194193e8acca01185a0f72bf4dac3eaa64d03fc082feeac1543f9c6291 WatchSource:0}: Error finding container 78e4a8194193e8acca01185a0f72bf4dac3eaa64d03fc082feeac1543f9c6291: Status 404 returned error can't find the container with id 78e4a8194193e8acca01185a0f72bf4dac3eaa64d03fc082feeac1543f9c6291
Sep 18 22:37:24.176477 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.176434    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8" event={"ID":"a8e05e15-28de-404b-8e84-e9ad5fbebcd1","Type":"ContainerStarted","Data":"33109b36579c263ae7d714f1552ac0465d5353919081032d6b642a020016d9ae"}
Sep 18 22:37:24.176941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.176498    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8" event={"ID":"a8e05e15-28de-404b-8e84-e9ad5fbebcd1","Type":"ContainerStarted","Data":"78e4a8194193e8acca01185a0f72bf4dac3eaa64d03fc082feeac1543f9c6291"}
Sep 18 22:37:24.176941 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.176528    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:24.178331 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.178295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl" event={"ID":"f5342023-b122-43e8-aa1d-00c093d4ae5b","Type":"ContainerStarted","Data":"fde3b8b0056735de7e40c60bff8910784db6005c6dc8ba75c4d6faab4ffb97b4"}
Sep 18 22:37:24.178491 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.178350    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:24.178491 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.178368    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl" event={"ID":"f5342023-b122-43e8-aa1d-00c093d4ae5b","Type":"ContainerStarted","Data":"bfe334fff2fe1a24613061a7acf3db3527688e12ca6db3bc2dee367dcbe0d99e"}
Sep 18 22:37:24.179910 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.179882    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" event={"ID":"db4fad39-1998-4a2c-85c8-545da2eafb75","Type":"ContainerStarted","Data":"c3524555b308308e1e84cb9142e3ba9cfaaf340ffcb19d6df5c0f56d0ffe5e8e"}
Sep 18 22:37:24.180045 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.179924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" event={"ID":"db4fad39-1998-4a2c-85c8-545da2eafb75","Type":"ContainerStarted","Data":"bd6b3af419b042cd1089a133b83cc781dc4f455614b8723ee42ece63a20f3613"}
Sep 18 22:37:24.180045 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.179998    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:24.194718 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.194674    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8" podStartSLOduration=2.194659796 podStartE2EDuration="2.194659796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:37:24.193219244 +0000 UTC m=+1228.553736103" watchObservedRunningTime="2026-09-18 22:37:24.194659796 +0000 UTC m=+1228.555176652"
Sep 18 22:37:24.217945 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.217900    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl" podStartSLOduration=2.217884822 podStartE2EDuration="2.217884822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:37:24.216773943 +0000 UTC m=+1228.577290825" watchObservedRunningTime="2026-09-18 22:37:24.217884822 +0000 UTC m=+1228.578401679"
Sep 18 22:37:24.253761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:24.253710    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" podStartSLOduration=1.941794611 podStartE2EDuration="2.253695s" totalImagesPullingTime="311.900389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:37:24.251254515 +0000 UTC m=+1228.611771373" watchObservedRunningTime="2026-09-18 22:37:24.253695 +0000 UTC m=+1228.614211857"
Sep 18 22:37:30.191564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:30.191531    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:37:35.188182 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:35.188144    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:37:35.188605 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:35.188221    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:37:57.791849 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:37:57.791771    2583 scope.go:117] "RemoveContainer" containerID="f0189af619c954c68ee8982eb9b029d3aa9da5b2623d1e22b43355af108a179a"
Sep 18 22:38:57.816714 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:38:57.816684    2583 scope.go:117] "RemoveContainer" containerID="bbb87f6ee2c66c92c8bd04114947cbe4ac6425ed74383bf065544366dfda2a82"
Sep 18 22:39:16.265500 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.265415    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"]
Sep 18 22:39:16.266083 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.265689    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f" podUID="f64261ca-eb23-4033-9fc6-49fd88a0c4d9" containerName="maas-api" containerID="cri-o://985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf" gracePeriod=30
Sep 18 22:39:16.372133 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.372092    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"]
Sep 18 22:39:16.372357 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.372331    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp" podUID="7525fdfb-e9e1-470b-9e09-3292855dde8e" containerName="payload-processing" containerID="cri-o://057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb" gracePeriod=30
Sep 18 22:39:16.386521 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.386486    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"]
Sep 18 22:39:16.386828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.386769    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd" podUID="4fe84c3a-eebc-4b18-9285-8774b351cb73" containerName="payload-pre-processing" containerID="cri-o://4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94" gracePeriod=30
Sep 18 22:39:16.544714 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.544427    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:39:16.629001 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.628976    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:39:16.631060 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.631037    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-openshift-service-ca\") pod \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") "
Sep 18 22:39:16.631177 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.631073    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-maas-api-tls\") pod \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") "
Sep 18 22:39:16.631255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.631189    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qph4h\" (UniqueName: \"kubernetes.io/projected/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-kube-api-access-qph4h\") pod \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") "
Sep 18 22:39:16.631255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.631231    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-metrics-certs\") pod \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\" (UID: \"f64261ca-eb23-4033-9fc6-49fd88a0c4d9\") "
Sep 18 22:39:16.631443 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.631414    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f64261ca-eb23-4033-9fc6-49fd88a0c4d9" (UID: "f64261ca-eb23-4033-9fc6-49fd88a0c4d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:39:16.631616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.631585    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.633058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.633031    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f64261ca-eb23-4033-9fc6-49fd88a0c4d9" (UID: "f64261ca-eb23-4033-9fc6-49fd88a0c4d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:39:16.633304 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.633271    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-kube-api-access-qph4h" (OuterVolumeSpecName: "kube-api-access-qph4h") pod "f64261ca-eb23-4033-9fc6-49fd88a0c4d9" (UID: "f64261ca-eb23-4033-9fc6-49fd88a0c4d9"). InnerVolumeSpecName "kube-api-access-qph4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:39:16.633427 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.633304    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f64261ca-eb23-4033-9fc6-49fd88a0c4d9" (UID: "f64261ca-eb23-4033-9fc6-49fd88a0c4d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:39:16.677176 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.677147    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:39:16.732735 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.732698    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk95m\" (UniqueName: \"kubernetes.io/projected/7525fdfb-e9e1-470b-9e09-3292855dde8e-kube-api-access-hk95m\") pod \"7525fdfb-e9e1-470b-9e09-3292855dde8e\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") "
Sep 18 22:39:16.732931 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.732849    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7525fdfb-e9e1-470b-9e09-3292855dde8e-plugins-config-volume\") pod \"7525fdfb-e9e1-470b-9e09-3292855dde8e\" (UID: \"7525fdfb-e9e1-470b-9e09-3292855dde8e\") "
Sep 18 22:39:16.733121 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.733104    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qph4h\" (UniqueName: \"kubernetes.io/projected/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-kube-api-access-qph4h\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.733186 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.733123    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.733186 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.733132    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64261ca-eb23-4033-9fc6-49fd88a0c4d9-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.733254 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.733228    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7525fdfb-e9e1-470b-9e09-3292855dde8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7525fdfb-e9e1-470b-9e09-3292855dde8e" (UID: "7525fdfb-e9e1-470b-9e09-3292855dde8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:39:16.735352 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.735324    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7525fdfb-e9e1-470b-9e09-3292855dde8e-kube-api-access-hk95m" (OuterVolumeSpecName: "kube-api-access-hk95m") pod "7525fdfb-e9e1-470b-9e09-3292855dde8e" (UID: "7525fdfb-e9e1-470b-9e09-3292855dde8e"). InnerVolumeSpecName "kube-api-access-hk95m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:39:16.834433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.834294    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtt4g\" (UniqueName: \"kubernetes.io/projected/4fe84c3a-eebc-4b18-9285-8774b351cb73-kube-api-access-dtt4g\") pod \"4fe84c3a-eebc-4b18-9285-8774b351cb73\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") "
Sep 18 22:39:16.834433 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.834373    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe84c3a-eebc-4b18-9285-8774b351cb73-plugins-config-volume\") pod \"4fe84c3a-eebc-4b18-9285-8774b351cb73\" (UID: \"4fe84c3a-eebc-4b18-9285-8774b351cb73\") "
Sep 18 22:39:16.834688 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.834636    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7525fdfb-e9e1-470b-9e09-3292855dde8e-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.834688 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.834650    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk95m\" (UniqueName: \"kubernetes.io/projected/7525fdfb-e9e1-470b-9e09-3292855dde8e-kube-api-access-hk95m\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.834819 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.834791    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fe84c3a-eebc-4b18-9285-8774b351cb73-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4fe84c3a-eebc-4b18-9285-8774b351cb73" (UID: "4fe84c3a-eebc-4b18-9285-8774b351cb73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:39:16.836416 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.836376    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fe84c3a-eebc-4b18-9285-8774b351cb73-kube-api-access-dtt4g" (OuterVolumeSpecName: "kube-api-access-dtt4g") pod "4fe84c3a-eebc-4b18-9285-8774b351cb73" (UID: "4fe84c3a-eebc-4b18-9285-8774b351cb73"). InnerVolumeSpecName "kube-api-access-dtt4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:39:16.856285 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.856252    2583 generic.go:356] "Generic (PLEG): container finished" podID="4fe84c3a-eebc-4b18-9285-8774b351cb73" containerID="4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94" exitCode=0
Sep 18 22:39:16.856479 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.856298    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"
Sep 18 22:39:16.856479 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.856335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd" event={"ID":"4fe84c3a-eebc-4b18-9285-8774b351cb73","Type":"ContainerDied","Data":"4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94"}
Sep 18 22:39:16.856479 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.856424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd" event={"ID":"4fe84c3a-eebc-4b18-9285-8774b351cb73","Type":"ContainerDied","Data":"4f61df993e97ef82fa966c1102cbce5398a28df10ae1a87afa5b5c4d0c89978e"}
Sep 18 22:39:16.856479 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.856458    2583 scope.go:117] "RemoveContainer" containerID="4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94"
Sep 18 22:39:16.857793 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.857770    2583 generic.go:356] "Generic (PLEG): container finished" podID="7525fdfb-e9e1-470b-9e09-3292855dde8e" containerID="057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb" exitCode=0
Sep 18 22:39:16.857857 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.857803    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"
Sep 18 22:39:16.857905 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.857874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp" event={"ID":"7525fdfb-e9e1-470b-9e09-3292855dde8e","Type":"ContainerDied","Data":"057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb"}
Sep 18 22:39:16.857954 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.857911    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp" event={"ID":"7525fdfb-e9e1-470b-9e09-3292855dde8e","Type":"ContainerDied","Data":"3219e3f30c89f17c79b8d7a97f19c2e5a72b0f8489e2ef34a0eb321db7bc79ad"}
Sep 18 22:39:16.859352 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.859329    2583 generic.go:356] "Generic (PLEG): container finished" podID="f64261ca-eb23-4033-9fc6-49fd88a0c4d9" containerID="985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf" exitCode=0
Sep 18 22:39:16.859498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.859380    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"
Sep 18 22:39:16.859498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.859418    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f" event={"ID":"f64261ca-eb23-4033-9fc6-49fd88a0c4d9","Type":"ContainerDied","Data":"985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf"}
Sep 18 22:39:16.859498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.859468    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f" event={"ID":"f64261ca-eb23-4033-9fc6-49fd88a0c4d9","Type":"ContainerDied","Data":"b81a2defbeb2dcfef8713684a8ab6cc863cee6ed0890d38dfa6e7c86d3c24a85"}
Sep 18 22:39:16.867420 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.867355    2583 scope.go:117] "RemoveContainer" containerID="4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94"
Sep 18 22:39:16.867783 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:39:16.867750    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94\": container with ID starting with 4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94 not found: ID does not exist" containerID="4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94"
Sep 18 22:39:16.867890 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.867790    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94"} err="failed to get container status \"4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94\": rpc error: code = NotFound desc = could not find container \"4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94\": container with ID starting with 4bee6e6316c676adc01f66d710097dc4da1ed1a871ce1066644f355b2ee04b94 not found: ID does not exist"
Sep 18 22:39:16.867890 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.867813    2583 scope.go:117] "RemoveContainer" containerID="057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb"
Sep 18 22:39:16.876795 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.876778    2583 scope.go:117] "RemoveContainer" containerID="057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb"
Sep 18 22:39:16.877051 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:39:16.877033    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb\": container with ID starting with 057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb not found: ID does not exist" containerID="057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb"
Sep 18 22:39:16.877087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.877059    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb"} err="failed to get container status \"057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb\": rpc error: code = NotFound desc = could not find container \"057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb\": container with ID starting with 057d00b43600fdf3d24bfd78e8483d0e805c5364c92d03a8ace6bca0a476d3bb not found: ID does not exist"
Sep 18 22:39:16.877087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.877073    2583 scope.go:117] "RemoveContainer" containerID="985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf"
Sep 18 22:39:16.884307 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.884280    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"]
Sep 18 22:39:16.885498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.885482    2583 scope.go:117] "RemoveContainer" containerID="985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf"
Sep 18 22:39:16.885735 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:39:16.885721    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf\": container with ID starting with 985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf not found: ID does not exist" containerID="985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf"
Sep 18 22:39:16.885785 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.885741    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf"} err="failed to get container status \"985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf\": rpc error: code = NotFound desc = could not find container \"985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf\": container with ID starting with 985ce02d2168787b96cba5d662fa0e7aa7dadb168c9c87b93c3de34a7fe332bf not found: ID does not exist"
Sep 18 22:39:16.889994 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.889973    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20e1128a-6896d4f574-z7ggd"]
Sep 18 22:39:16.901643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.901621    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"]
Sep 18 22:39:16.903902 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.903882    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-20e1128a-5ff9b5c65d-rtd6f"]
Sep 18 22:39:16.917058 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.917027    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"]
Sep 18 22:39:16.919471 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.919448    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20e1128a-7f4d7ff4b5-blrtp"]
Sep 18 22:39:16.935253 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.935228    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtt4g\" (UniqueName: \"kubernetes.io/projected/4fe84c3a-eebc-4b18-9285-8774b351cb73-kube-api-access-dtt4g\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:16.935253 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:16.935249    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fe84c3a-eebc-4b18-9285-8774b351cb73-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:39:17.683707 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:17.683673    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fe84c3a-eebc-4b18-9285-8774b351cb73" path="/var/lib/kubelet/pods/4fe84c3a-eebc-4b18-9285-8774b351cb73/volumes"
Sep 18 22:39:17.684103 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:17.684088    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7525fdfb-e9e1-470b-9e09-3292855dde8e" path="/var/lib/kubelet/pods/7525fdfb-e9e1-470b-9e09-3292855dde8e/volumes"
Sep 18 22:39:17.684457 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:17.684445    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64261ca-eb23-4033-9fc6-49fd88a0c4d9" path="/var/lib/kubelet/pods/f64261ca-eb23-4033-9fc6-49fd88a0c4d9/volumes"
Sep 18 22:39:42.093705 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.093659    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"]
Sep 18 22:39:42.094131 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094116    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7525fdfb-e9e1-470b-9e09-3292855dde8e" containerName="payload-processing"
Sep 18 22:39:42.094175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094133    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7525fdfb-e9e1-470b-9e09-3292855dde8e" containerName="payload-processing"
Sep 18 22:39:42.094175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094146    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fe84c3a-eebc-4b18-9285-8774b351cb73" containerName="payload-pre-processing"
Sep 18 22:39:42.094175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094151    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fe84c3a-eebc-4b18-9285-8774b351cb73" containerName="payload-pre-processing"
Sep 18 22:39:42.094175 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094173    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64261ca-eb23-4033-9fc6-49fd88a0c4d9" containerName="maas-api"
Sep 18 22:39:42.094297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094179    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64261ca-eb23-4033-9fc6-49fd88a0c4d9" containerName="maas-api"
Sep 18 22:39:42.094297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094241    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7525fdfb-e9e1-470b-9e09-3292855dde8e" containerName="payload-processing"
Sep 18 22:39:42.094297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094250    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64261ca-eb23-4033-9fc6-49fd88a0c4d9" containerName="maas-api"
Sep 18 22:39:42.094297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.094257    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fe84c3a-eebc-4b18-9285-8774b351cb73" containerName="payload-pre-processing"
Sep 18 22:39:42.098707 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.098687    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.101630 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.101600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a91c21\"/\"auth-test-model-a91c21-kserve-self-signed-certs\""
Sep 18 22:39:42.102539 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.102514    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a91c21\"/\"default-dockercfg-vphth\""
Sep 18 22:39:42.102657 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.102518    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a91c21\"/\"openshift-service-ca.crt\""
Sep 18 22:39:42.119856 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.119822    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"]
Sep 18 22:39:42.121236 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.121214    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a91c21\"/\"kube-root-ca.crt\""
Sep 18 22:39:42.168759 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.168722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-home\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.168759 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.168760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-dshm\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.169016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.168865    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tq7s\" (UniqueName: \"kubernetes.io/projected/731d4315-b4e2-4b01-9f83-0a5433616ed7-kube-api-access-5tq7s\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.169016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.168902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-tmp-dir\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.169016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.168981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-model-cache\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.169016 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.169009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731d4315-b4e2-4b01-9f83-0a5433616ed7-tls-certs\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.271174 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.271091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tq7s\" (UniqueName: \"kubernetes.io/projected/731d4315-b4e2-4b01-9f83-0a5433616ed7-kube-api-access-5tq7s\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.271174 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.271158    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-tmp-dir\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.271514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.271204    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-model-cache\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.271514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.271229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731d4315-b4e2-4b01-9f83-0a5433616ed7-tls-certs\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.271514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.271299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-home\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.271514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.271499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-dshm\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.272216 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.272194    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-model-cache\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.272320 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.272228    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-tmp-dir\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.272374 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.272338    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-home\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.274638 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.274598    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-dshm\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.275204 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.275183    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731d4315-b4e2-4b01-9f83-0a5433616ed7-tls-certs\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.284496 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.284468    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tq7s\" (UniqueName: \"kubernetes.io/projected/731d4315-b4e2-4b01-9f83-0a5433616ed7-kube-api-access-5tq7s\") pod \"auth-test-model-a91c21-kserve-678bcb865f-7r5p7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") " pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.408578 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.408547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:42.693525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:42.693500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"]
Sep 18 22:39:42.695618 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:39:42.695589    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod731d4315_b4e2_4b01_9f83_0a5433616ed7.slice/crio-b79376cdd30ff698dadbc024e0026eaa38a7acdf698fb33ee0af16695ab3fcb0 WatchSource:0}: Error finding container b79376cdd30ff698dadbc024e0026eaa38a7acdf698fb33ee0af16695ab3fcb0: Status 404 returned error can't find the container with id b79376cdd30ff698dadbc024e0026eaa38a7acdf698fb33ee0af16695ab3fcb0
Sep 18 22:39:43.025953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:43.025860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7" event={"ID":"731d4315-b4e2-4b01-9f83-0a5433616ed7","Type":"ContainerStarted","Data":"235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222"}
Sep 18 22:39:43.025953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:43.025899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7" event={"ID":"731d4315-b4e2-4b01-9f83-0a5433616ed7","Type":"ContainerStarted","Data":"b79376cdd30ff698dadbc024e0026eaa38a7acdf698fb33ee0af16695ab3fcb0"}
Sep 18 22:39:43.063747 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:43.063686    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7" podStartSLOduration=1.063669621 podStartE2EDuration="1.063669621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:39:43.060188956 +0000 UTC m=+1367.420705826" watchObservedRunningTime="2026-09-18 22:39:43.063669621 +0000 UTC m=+1367.424186478"
Sep 18 22:39:52.409572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:52.409518    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:52.409572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:52.409590    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:52.422439 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:52.422409    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:39:53.094067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:39:53.094040    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:41:28.349907 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.349868    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"]
Sep 18 22:41:28.352925 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.352909    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.355865 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.355841    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-37cf0f\""
Sep 18 22:41:28.374379 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.374350    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"]
Sep 18 22:41:28.421767 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.421731    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"]
Sep 18 22:41:28.425290 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.425274    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.428189 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.428166    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-37cf0f-dockercfg-dhvh4\""
Sep 18 22:41:28.455515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.455490    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"]
Sep 18 22:41:28.455668 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.455578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxrwc\" (UniqueName: \"kubernetes.io/projected/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-kube-api-access-pxrwc\") pod \"payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") " pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.455668 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.455641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-plugins-config-volume\") pod \"payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") " pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.556303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.556266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.556510 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.556319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxrwc\" (UniqueName: \"kubernetes.io/projected/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-kube-api-access-pxrwc\") pod \"payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") " pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.556510 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.556366    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-plugins-config-volume\") pod \"payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") " pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.556510 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.556473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t6c2\" (UniqueName: \"kubernetes.io/projected/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-kube-api-access-5t6c2\") pod \"payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.557067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.557043    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-plugins-config-volume\") pod \"payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") " pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.564891 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.564860    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxrwc\" (UniqueName: \"kubernetes.io/projected/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-kube-api-access-pxrwc\") pod \"payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") " pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.657400 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.657363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.657592 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.657475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t6c2\" (UniqueName: \"kubernetes.io/projected/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-kube-api-access-5t6c2\") pod \"payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.658220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.658198    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.663031 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.663007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:28.665421 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.665381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t6c2\" (UniqueName: \"kubernetes.io/projected/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-kube-api-access-5t6c2\") pod \"payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.735636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.735607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:28.937590 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:28.937563    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"]
Sep 18 22:41:28.938851 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:41:28.938825    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ac49ea_4f3b_477f_8fb3_403d5ae45fae.slice/crio-a77b923ea898dd79628077155153cbc99054681992270b4b4232829c873f3acd WatchSource:0}: Error finding container a77b923ea898dd79628077155153cbc99054681992270b4b4232829c873f3acd: Status 404 returned error can't find the container with id a77b923ea898dd79628077155153cbc99054681992270b4b4232829c873f3acd
Sep 18 22:41:29.016539 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.016515    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"]
Sep 18 22:41:29.017410 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:41:29.017365    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ba5ac0_d0f4_47e7_8a54_ff1f6a53e427.slice/crio-a37e07187fc4ccc9c21f144d386288f00214fa8a2345771644a59803d6f17768 WatchSource:0}: Error finding container a37e07187fc4ccc9c21f144d386288f00214fa8a2345771644a59803d6f17768: Status 404 returned error can't find the container with id a37e07187fc4ccc9c21f144d386288f00214fa8a2345771644a59803d6f17768
Sep 18 22:41:29.660111 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.660067    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf" event={"ID":"41ac49ea-4f3b-477f-8fb3-403d5ae45fae","Type":"ContainerStarted","Data":"eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2"}
Sep 18 22:41:29.660616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.660139    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:29.660616 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.660164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf" event={"ID":"41ac49ea-4f3b-477f-8fb3-403d5ae45fae","Type":"ContainerStarted","Data":"a77b923ea898dd79628077155153cbc99054681992270b4b4232829c873f3acd"}
Sep 18 22:41:29.661970 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.661936    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" event={"ID":"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427","Type":"ContainerStarted","Data":"4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927"}
Sep 18 22:41:29.662100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.661986    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:29.662100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.662003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" event={"ID":"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427","Type":"ContainerStarted","Data":"a37e07187fc4ccc9c21f144d386288f00214fa8a2345771644a59803d6f17768"}
Sep 18 22:41:29.700022 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.699973    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf" podStartSLOduration=1.699956817 podStartE2EDuration="1.699956817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:41:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:41:29.697947806 +0000 UTC m=+1474.058464664" watchObservedRunningTime="2026-09-18 22:41:29.699956817 +0000 UTC m=+1474.060473676"
Sep 18 22:41:29.732910 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:29.732854    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" podStartSLOduration=1.732836377 podStartE2EDuration="1.732836377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:41:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:41:29.730286369 +0000 UTC m=+1474.090803227" watchObservedRunningTime="2026-09-18 22:41:29.732836377 +0000 UTC m=+1474.093353235"
Sep 18 22:41:40.669051 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:40.669022    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:41:40.669576 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:40.669095    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:41:57.706500 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:57.706472    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:41:57.708745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:57.708721    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:41:58.014927 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.014844    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"]
Sep 18 22:41:58.018782 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.018759    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.021823 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.021801    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d90856-serving-cert\""
Sep 18 22:41:58.034171 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.034140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"]
Sep 18 22:41:58.143017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.142969    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhqzp\" (UniqueName: \"kubernetes.io/projected/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-kube-api-access-hhqzp\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.143017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.143010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.143258 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.143032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-metrics-certs\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.143258 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.143175    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-openshift-service-ca\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.244201 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.244165    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhqzp\" (UniqueName: \"kubernetes.io/projected/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-kube-api-access-hhqzp\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.244431 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.244206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.244431 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.244248    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-metrics-certs\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.244431 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.244295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-openshift-service-ca\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.244681 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:41:58.244483    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-serving-cert: secret "maas-api-e2e-api-b-d90856-serving-cert" not found
Sep 18 22:41:58.244681 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:41:58.244584    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls podName:a9640f3e-1a57-4535-aa72-ffe1fc71cfbe nodeName:}" failed. No retries permitted until 2026-09-18 22:41:58.744562359 +0000 UTC m=+1503.105079203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls") pod "maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" (UID: "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe") : secret "maas-api-e2e-api-b-d90856-serving-cert" not found
Sep 18 22:41:58.245077 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.245058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-openshift-service-ca\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.246926 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.246899    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-metrics-certs\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.255532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.255511    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhqzp\" (UniqueName: \"kubernetes.io/projected/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-kube-api-access-hhqzp\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.750101 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.750057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.752447 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.752424    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls\") pod \"maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.871032 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.870994    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"]
Sep 18 22:41:58.874673 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.874651    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:58.880611 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.880588    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d90856\""
Sep 18 22:41:58.906814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.906773    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"]
Sep 18 22:41:58.930654 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.930617    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:58.951617 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.951587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d9dd7a-6198-4228-b827-35482130c7b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:58.951749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:58.951646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8b2j\" (UniqueName: \"kubernetes.io/projected/93d9dd7a-6198-4228-b827-35482130c7b4-kube-api-access-x8b2j\") pod \"payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.053002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.052906    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d9dd7a-6198-4228-b827-35482130c7b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.053002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.052973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8b2j\" (UniqueName: \"kubernetes.io/projected/93d9dd7a-6198-4228-b827-35482130c7b4-kube-api-access-x8b2j\") pod \"payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.053621 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.053601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d9dd7a-6198-4228-b827-35482130c7b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.066502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.066480    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8b2j\" (UniqueName: \"kubernetes.io/projected/93d9dd7a-6198-4228-b827-35482130c7b4-kube-api-access-x8b2j\") pod \"payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.187151 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.187111    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.217047 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:41:59.216997    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9640f3e_1a57_4535_aa72_ffe1fc71cfbe.slice/crio-57c1b38d68fb5b062fdf54cc1f47371c325f9714d46350379bb9651482281b27 WatchSource:0}: Error finding container 57c1b38d68fb5b062fdf54cc1f47371c325f9714d46350379bb9651482281b27: Status 404 returned error can't find the container with id 57c1b38d68fb5b062fdf54cc1f47371c325f9714d46350379bb9651482281b27
Sep 18 22:41:59.218108 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.218078    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"]
Sep 18 22:41:59.218873 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.218853    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:41:59.497757 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.497730    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"]
Sep 18 22:41:59.499074 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:41:59.499045    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d9dd7a_6198_4228_b827_35482130c7b4.slice/crio-72c929a143e23dbe72c5535f78e2e2cddeae41a5f48a5003de4d221f01e374a2 WatchSource:0}: Error finding container 72c929a143e23dbe72c5535f78e2e2cddeae41a5f48a5003de4d221f01e374a2: Status 404 returned error can't find the container with id 72c929a143e23dbe72c5535f78e2e2cddeae41a5f48a5003de4d221f01e374a2
Sep 18 22:41:59.839319 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.839228    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn" event={"ID":"93d9dd7a-6198-4228-b827-35482130c7b4","Type":"ContainerStarted","Data":"51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28"}
Sep 18 22:41:59.839319 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.839288    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn" event={"ID":"93d9dd7a-6198-4228-b827-35482130c7b4","Type":"ContainerStarted","Data":"72c929a143e23dbe72c5535f78e2e2cddeae41a5f48a5003de4d221f01e374a2"}
Sep 18 22:41:59.839839 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.839410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:41:59.841063 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.841038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" event={"ID":"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe","Type":"ContainerStarted","Data":"b024ce7a388580bc175db4395ab5c77b7841a03b141086b9d1ee662914ff28e3"}
Sep 18 22:41:59.841208 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.841077    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" event={"ID":"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe","Type":"ContainerStarted","Data":"57c1b38d68fb5b062fdf54cc1f47371c325f9714d46350379bb9651482281b27"}
Sep 18 22:41:59.841208 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.841116    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:41:59.858662 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.858617    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn" podStartSLOduration=1.8586027619999999 podStartE2EDuration="1.858602762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:41:59.856188367 +0000 UTC m=+1504.216705225" watchObservedRunningTime="2026-09-18 22:41:59.858602762 +0000 UTC m=+1504.219119618"
Sep 18 22:41:59.879011 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:41:59.878963    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" podStartSLOduration=2.539023915 podStartE2EDuration="2.878948291s" totalImagesPullingTime="339.924376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:41:59.878235631 +0000 UTC m=+1504.238752489" watchObservedRunningTime="2026-09-18 22:41:59.878948291 +0000 UTC m=+1504.239465147"
Sep 18 22:42:05.854219 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:05.854191    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:42:10.850954 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:10.850878    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:42:33.280617 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.280578    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"]
Sep 18 22:42:33.283814 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.283787    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-c5fdd3-29829510-rkwr6"]
Sep 18 22:42:33.313364 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.313326    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"]
Sep 18 22:42:33.313648 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.313589    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc" podUID="d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" containerName="payload-pre-processing" containerID="cri-o://59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7" gracePeriod=30
Sep 18 22:42:33.584799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.584756    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:42:33.683864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.683831    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5465553c-3f51-4668-bd8e-5f100cd66f61" path="/var/lib/kubelet/pods/5465553c-3f51-4668-bd8e-5f100cd66f61/volumes"
Sep 18 22:42:33.698011 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.697985    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4c6v\" (UniqueName: \"kubernetes.io/projected/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-kube-api-access-z4c6v\") pod \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") "
Sep 18 22:42:33.698166 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.698126    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-plugins-config-volume\") pod \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\" (UID: \"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0\") "
Sep 18 22:42:33.698450 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.698426    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" (UID: "d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:42:33.700027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.700008    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-kube-api-access-z4c6v" (OuterVolumeSpecName: "kube-api-access-z4c6v") pod "d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" (UID: "d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0"). InnerVolumeSpecName "kube-api-access-z4c6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:42:33.799800 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.799720    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4c6v\" (UniqueName: \"kubernetes.io/projected/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-kube-api-access-z4c6v\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:42:33.799800 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:33.799746    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:42:34.052862 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.052767    2583 generic.go:356] "Generic (PLEG): container finished" podID="d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" containerID="59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7" exitCode=0
Sep 18 22:42:34.052862 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.052810    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"
Sep 18 22:42:34.053069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.052854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc" event={"ID":"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0","Type":"ContainerDied","Data":"59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7"}
Sep 18 22:42:34.053069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.052907    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc" event={"ID":"d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0","Type":"ContainerDied","Data":"32a0ff9d9edc1dee46c252498e05719c36bd15606f75be26d9ff40059cb39429"}
Sep 18 22:42:34.053069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.052930    2583 scope.go:117] "RemoveContainer" containerID="59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7"
Sep 18 22:42:34.063183 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.063162    2583 scope.go:117] "RemoveContainer" containerID="59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7"
Sep 18 22:42:34.063570 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:34.063535    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7\": container with ID starting with 59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7 not found: ID does not exist" containerID="59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7"
Sep 18 22:42:34.063683 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.063591    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7"} err="failed to get container status \"59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7\": rpc error: code = NotFound desc = could not find container \"59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7\": container with ID starting with 59f75f99dbc768bbe39205257a06e585c9824133b027eb0e324ca5713af27da7 not found: ID does not exist"
Sep 18 22:42:34.077033 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.077011    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"]
Sep 18 22:42:34.082738 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:34.082715    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c5fdd3-77674d5b94-c7vjc"]
Sep 18 22:42:35.683004 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:35.682970    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" path="/var/lib/kubelet/pods/d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0/volumes"
Sep 18 22:42:43.722533 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:43.722498    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"]
Sep 18 22:42:43.723071 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:43.722730    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf" podUID="41ac49ea-4f3b-477f-8fb3-403d5ae45fae" containerName="payload-processing" containerID="cri-o://eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2" gracePeriod=30
Sep 18 22:42:43.731813 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:43.731359    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"]
Sep 18 22:42:43.807335 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:43.807300    2583 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-api-a-37cf0f: configmap "payload-processing-plugins-e2e-api-a-37cf0f" not found
Sep 18 22:42:43.807517 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:43.807428    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume podName:64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427 nodeName:}" failed. No retries permitted until 2026-09-18 22:42:44.307403534 +0000 UTC m=+1548.667920391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume") pod "payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" (UID: "64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427") : configmap "payload-processing-plugins-e2e-api-a-37cf0f" not found
Sep 18 22:42:43.987683 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:43.987622    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:42:44.110623 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.110585    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-plugins-config-volume\") pod \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") "
Sep 18 22:42:44.110820 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.110673    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxrwc\" (UniqueName: \"kubernetes.io/projected/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-kube-api-access-pxrwc\") pod \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\" (UID: \"41ac49ea-4f3b-477f-8fb3-403d5ae45fae\") "
Sep 18 22:42:44.111038 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.111003    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41ac49ea-4f3b-477f-8fb3-403d5ae45fae" (UID: "41ac49ea-4f3b-477f-8fb3-403d5ae45fae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:42:44.112971 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.112945    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-kube-api-access-pxrwc" (OuterVolumeSpecName: "kube-api-access-pxrwc") pod "41ac49ea-4f3b-477f-8fb3-403d5ae45fae" (UID: "41ac49ea-4f3b-477f-8fb3-403d5ae45fae"). InnerVolumeSpecName "kube-api-access-pxrwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:42:44.114862 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.114838    2583 generic.go:356] "Generic (PLEG): container finished" podID="41ac49ea-4f3b-477f-8fb3-403d5ae45fae" containerID="eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2" exitCode=0
Sep 18 22:42:44.114959 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.114886    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"
Sep 18 22:42:44.115015 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.114961    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf" event={"ID":"41ac49ea-4f3b-477f-8fb3-403d5ae45fae","Type":"ContainerDied","Data":"eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2"}
Sep 18 22:42:44.115067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.115020    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf" event={"ID":"41ac49ea-4f3b-477f-8fb3-403d5ae45fae","Type":"ContainerDied","Data":"a77b923ea898dd79628077155153cbc99054681992270b4b4232829c873f3acd"}
Sep 18 22:42:44.115137 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.115123    2583 scope.go:117] "RemoveContainer" containerID="eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2"
Sep 18 22:42:44.115198 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.115152    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" podUID="64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" containerName="payload-pre-processing" containerID="cri-o://4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927" gracePeriod=30
Sep 18 22:42:44.130961 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.130934    2583 scope.go:117] "RemoveContainer" containerID="eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2"
Sep 18 22:42:44.131428 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:44.131367    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2\": container with ID starting with eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2 not found: ID does not exist" containerID="eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2"
Sep 18 22:42:44.131527 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.131425    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2"} err="failed to get container status \"eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2\": rpc error: code = NotFound desc = could not find container \"eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2\": container with ID starting with eaac372b8331d48e3a1cc9708e7caab6d4f18f0e50163d55f951ad44a0fcbac2 not found: ID does not exist"
Sep 18 22:42:44.191330 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.191298    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"]
Sep 18 22:42:44.197645 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.197620    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-37cf0f-f9dfc6fbf-jnhzf"]
Sep 18 22:42:44.211414 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.211361    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:42:44.211414 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.211398    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxrwc\" (UniqueName: \"kubernetes.io/projected/41ac49ea-4f3b-477f-8fb3-403d5ae45fae-kube-api-access-pxrwc\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:42:44.312072 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:44.312044    2583 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-api-a-37cf0f: configmap "payload-processing-plugins-e2e-api-a-37cf0f" not found
Sep 18 22:42:44.312273 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:44.312262    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume podName:64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427 nodeName:}" failed. No retries permitted until 2026-09-18 22:42:45.312246276 +0000 UTC m=+1549.672763111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume") pod "payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" (UID: "64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427") : configmap "payload-processing-plugins-e2e-api-a-37cf0f" not found
Sep 18 22:42:44.382750 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.382718    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:42:44.513301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.513268    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t6c2\" (UniqueName: \"kubernetes.io/projected/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-kube-api-access-5t6c2\") pod \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") "
Sep 18 22:42:44.513301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.513307    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume\") pod \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\" (UID: \"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427\") "
Sep 18 22:42:44.513751 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.513720    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" (UID: "64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:42:44.515435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.515410    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-kube-api-access-5t6c2" (OuterVolumeSpecName: "kube-api-access-5t6c2") pod "64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" (UID: "64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427"). InnerVolumeSpecName "kube-api-access-5t6c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:42:44.614656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.614574    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t6c2\" (UniqueName: \"kubernetes.io/projected/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-kube-api-access-5t6c2\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:42:44.614656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:44.614604    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:42:45.123139 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.123104    2583 generic.go:356] "Generic (PLEG): container finished" podID="64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" containerID="4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927" exitCode=0
Sep 18 22:42:45.123564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.123149    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"
Sep 18 22:42:45.123564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.123186    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" event={"ID":"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427","Type":"ContainerDied","Data":"4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927"}
Sep 18 22:42:45.123564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.123241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8" event={"ID":"64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427","Type":"ContainerDied","Data":"a37e07187fc4ccc9c21f144d386288f00214fa8a2345771644a59803d6f17768"}
Sep 18 22:42:45.123564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.123264    2583 scope.go:117] "RemoveContainer" containerID="4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927"
Sep 18 22:42:45.132938 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.132924    2583 scope.go:117] "RemoveContainer" containerID="4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927"
Sep 18 22:42:45.133204 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:42:45.133183    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927\": container with ID starting with 4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927 not found: ID does not exist" containerID="4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927"
Sep 18 22:42:45.133300 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.133208    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927"} err="failed to get container status \"4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927\": rpc error: code = NotFound desc = could not find container \"4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927\": container with ID starting with 4e8be08c40e3bc8272292f8da21ff3a7c44c0c42f63a66878bf99e3991a6d927 not found: ID does not exist"
Sep 18 22:42:45.150681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.150656    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"]
Sep 18 22:42:45.155180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.155151    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-37cf0f-69f548795d-zlwj8"]
Sep 18 22:42:45.684183 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.684146    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ac49ea-4f3b-477f-8fb3-403d5ae45fae" path="/var/lib/kubelet/pods/41ac49ea-4f3b-477f-8fb3-403d5ae45fae/volumes"
Sep 18 22:42:45.684768 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:45.684746    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" path="/var/lib/kubelet/pods/64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427/volumes"
Sep 18 22:42:57.881549 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:42:57.881515    2583 scope.go:117] "RemoveContainer" containerID="e672df7a5266d71534cb6cf4cd8648fde916c97ec89317efbd7ca38eeb6fb6eb"
Sep 18 22:43:07.115763 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.115729    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"]
Sep 18 22:43:07.116246 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.115983    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" podUID="a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" containerName="maas-api" containerID="cri-o://b024ce7a388580bc175db4395ab5c77b7841a03b141086b9d1ee662914ff28e3" gracePeriod=30
Sep 18 22:43:07.230929 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.230905    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"]
Sep 18 22:43:07.231252 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.231125    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn" podUID="93d9dd7a-6198-4228-b827-35482130c7b4" containerName="payload-pre-processing" containerID="cri-o://51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28" gracePeriod=30
Sep 18 22:43:07.266003 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.265968    2583 generic.go:356] "Generic (PLEG): container finished" podID="a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" containerID="b024ce7a388580bc175db4395ab5c77b7841a03b141086b9d1ee662914ff28e3" exitCode=0
Sep 18 22:43:07.268408 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.267746    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" event={"ID":"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe","Type":"ContainerDied","Data":"b024ce7a388580bc175db4395ab5c77b7841a03b141086b9d1ee662914ff28e3"}
Sep 18 22:43:07.401714 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.401681    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:43:07.435519 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.435486    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhqzp\" (UniqueName: \"kubernetes.io/projected/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-kube-api-access-hhqzp\") pod \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") "
Sep 18 22:43:07.435699 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.435579    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls\") pod \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") "
Sep 18 22:43:07.435699 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.435608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-metrics-certs\") pod \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") "
Sep 18 22:43:07.435699 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.435627    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-openshift-service-ca\") pod \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\" (UID: \"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe\") "
Sep 18 22:43:07.436069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.436038    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" (UID: "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:07.437939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.437908    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" (UID: "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:43:07.437939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.437924    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" (UID: "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:43:07.438136 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.438002    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-kube-api-access-hhqzp" (OuterVolumeSpecName: "kube-api-access-hhqzp") pod "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" (UID: "a9640f3e-1a57-4535-aa72-ffe1fc71cfbe"). InnerVolumeSpecName "kube-api-access-hhqzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:07.496911 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.496887    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:43:07.536477 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536446    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d9dd7a-6198-4228-b827-35482130c7b4-plugins-config-volume\") pod \"93d9dd7a-6198-4228-b827-35482130c7b4\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") "
Sep 18 22:43:07.536664 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536498    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8b2j\" (UniqueName: \"kubernetes.io/projected/93d9dd7a-6198-4228-b827-35482130c7b4-kube-api-access-x8b2j\") pod \"93d9dd7a-6198-4228-b827-35482130c7b4\" (UID: \"93d9dd7a-6198-4228-b827-35482130c7b4\") "
Sep 18 22:43:07.536830 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536811    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:07.536896 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536834    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:07.536896 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536852    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:07.536896 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536866    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhqzp\" (UniqueName: \"kubernetes.io/projected/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe-kube-api-access-hhqzp\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:07.536896 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.536856    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93d9dd7a-6198-4228-b827-35482130c7b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93d9dd7a-6198-4228-b827-35482130c7b4" (UID: "93d9dd7a-6198-4228-b827-35482130c7b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:07.538720 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.538696    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93d9dd7a-6198-4228-b827-35482130c7b4-kube-api-access-x8b2j" (OuterVolumeSpecName: "kube-api-access-x8b2j") pod "93d9dd7a-6198-4228-b827-35482130c7b4" (UID: "93d9dd7a-6198-4228-b827-35482130c7b4"). InnerVolumeSpecName "kube-api-access-x8b2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:07.637786 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.637750    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93d9dd7a-6198-4228-b827-35482130c7b4-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:07.637786 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:07.637785    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8b2j\" (UniqueName: \"kubernetes.io/projected/93d9dd7a-6198-4228-b827-35482130c7b4-kube-api-access-x8b2j\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:08.274969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.274938    2583 generic.go:356] "Generic (PLEG): container finished" podID="93d9dd7a-6198-4228-b827-35482130c7b4" containerID="51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28" exitCode=0
Sep 18 22:43:08.275441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.274992    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"
Sep 18 22:43:08.275441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.275011    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn" event={"ID":"93d9dd7a-6198-4228-b827-35482130c7b4","Type":"ContainerDied","Data":"51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28"}
Sep 18 22:43:08.275441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.275060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn" event={"ID":"93d9dd7a-6198-4228-b827-35482130c7b4","Type":"ContainerDied","Data":"72c929a143e23dbe72c5535f78e2e2cddeae41a5f48a5003de4d221f01e374a2"}
Sep 18 22:43:08.275441 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.275087    2583 scope.go:117] "RemoveContainer" containerID="51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28"
Sep 18 22:43:08.276570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.276429    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"
Sep 18 22:43:08.276570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.276468    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj" event={"ID":"a9640f3e-1a57-4535-aa72-ffe1fc71cfbe","Type":"ContainerDied","Data":"57c1b38d68fb5b062fdf54cc1f47371c325f9714d46350379bb9651482281b27"}
Sep 18 22:43:08.284832 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.284811    2583 scope.go:117] "RemoveContainer" containerID="51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28"
Sep 18 22:43:08.285096 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:43:08.285077    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28\": container with ID starting with 51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28 not found: ID does not exist" containerID="51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28"
Sep 18 22:43:08.285181 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.285103    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28"} err="failed to get container status \"51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28\": rpc error: code = NotFound desc = could not find container \"51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28\": container with ID starting with 51a2f01ddd39d4f2a8069bd5ba609705e0452cfad11de38ed3c7090794262f28 not found: ID does not exist"
Sep 18 22:43:08.285181 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.285120    2583 scope.go:117] "RemoveContainer" containerID="b024ce7a388580bc175db4395ab5c77b7841a03b141086b9d1ee662914ff28e3"
Sep 18 22:43:08.294994 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.294974    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"]
Sep 18 22:43:08.300520 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.300496    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d90856-7d494bbbb8-h94lj"]
Sep 18 22:43:08.311417 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.311366    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"]
Sep 18 22:43:08.314935 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:08.314913    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d90856-5779bc7854-8p4hn"]
Sep 18 22:43:09.683260 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:09.683225    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93d9dd7a-6198-4228-b827-35482130c7b4" path="/var/lib/kubelet/pods/93d9dd7a-6198-4228-b827-35482130c7b4/volumes"
Sep 18 22:43:09.683661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:09.683652    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" path="/var/lib/kubelet/pods/a9640f3e-1a57-4535-aa72-ffe1fc71cfbe/volumes"
Sep 18 22:43:24.696398 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:24.696344    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"]
Sep 18 22:43:24.697240 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:24.697165    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4" podUID="56e9fa0a-325e-4e37-a94c-c25726a26184" containerName="maas-api" containerID="cri-o://7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1" gracePeriod=30
Sep 18 22:43:24.760133 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:24.760095    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"]
Sep 18 22:43:24.760399 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:24.760356    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx" podUID="98535521-9eeb-4bfc-917f-185903823c43" containerName="payload-processing" containerID="cri-o://a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5" gracePeriod=30
Sep 18 22:43:24.768074 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:24.768039    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"]
Sep 18 22:43:24.768316 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:24.768268    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2" podUID="1809341b-65c6-4c63-a63c-33ba1e3a63d8" containerName="payload-pre-processing" containerID="cri-o://90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0" gracePeriod=30
Sep 18 22:43:25.045976 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.045940    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:43:25.075817 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.075787    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:43:25.086242 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.086220    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:43:25.208799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208767    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-metrics-certs\") pod \"56e9fa0a-325e-4e37-a94c-c25726a26184\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") "
Sep 18 22:43:25.208972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208822    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56e9fa0a-325e-4e37-a94c-c25726a26184-openshift-service-ca\") pod \"56e9fa0a-325e-4e37-a94c-c25726a26184\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") "
Sep 18 22:43:25.208972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208847    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2559\" (UniqueName: \"kubernetes.io/projected/1809341b-65c6-4c63-a63c-33ba1e3a63d8-kube-api-access-j2559\") pod \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") "
Sep 18 22:43:25.208972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208872    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98535521-9eeb-4bfc-917f-185903823c43-plugins-config-volume\") pod \"98535521-9eeb-4bfc-917f-185903823c43\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") "
Sep 18 22:43:25.208972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208902    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-maas-api-tls\") pod \"56e9fa0a-325e-4e37-a94c-c25726a26184\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") "
Sep 18 22:43:25.208972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208928    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rq65\" (UniqueName: \"kubernetes.io/projected/56e9fa0a-325e-4e37-a94c-c25726a26184-kube-api-access-6rq65\") pod \"56e9fa0a-325e-4e37-a94c-c25726a26184\" (UID: \"56e9fa0a-325e-4e37-a94c-c25726a26184\") "
Sep 18 22:43:25.208972 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.208944    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1809341b-65c6-4c63-a63c-33ba1e3a63d8-plugins-config-volume\") pod \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\" (UID: \"1809341b-65c6-4c63-a63c-33ba1e3a63d8\") "
Sep 18 22:43:25.209310 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209213    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jxnz\" (UniqueName: \"kubernetes.io/projected/98535521-9eeb-4bfc-917f-185903823c43-kube-api-access-9jxnz\") pod \"98535521-9eeb-4bfc-917f-185903823c43\" (UID: \"98535521-9eeb-4bfc-917f-185903823c43\") "
Sep 18 22:43:25.209364 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209304    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e9fa0a-325e-4e37-a94c-c25726a26184-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56e9fa0a-325e-4e37-a94c-c25726a26184" (UID: "56e9fa0a-325e-4e37-a94c-c25726a26184"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:25.209364 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209316    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1809341b-65c6-4c63-a63c-33ba1e3a63d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1809341b-65c6-4c63-a63c-33ba1e3a63d8" (UID: "1809341b-65c6-4c63-a63c-33ba1e3a63d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:25.209552 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209526    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98535521-9eeb-4bfc-917f-185903823c43-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98535521-9eeb-4bfc-917f-185903823c43" (UID: "98535521-9eeb-4bfc-917f-185903823c43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:25.209640 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209623    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56e9fa0a-325e-4e37-a94c-c25726a26184-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.209695 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209647    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98535521-9eeb-4bfc-917f-185903823c43-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.209695 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.209661    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1809341b-65c6-4c63-a63c-33ba1e3a63d8-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.211605 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.211579    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56e9fa0a-325e-4e37-a94c-c25726a26184" (UID: "56e9fa0a-325e-4e37-a94c-c25726a26184"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:43:25.211739 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.211622    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "56e9fa0a-325e-4e37-a94c-c25726a26184" (UID: "56e9fa0a-325e-4e37-a94c-c25726a26184"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:43:25.211739 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.211698    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98535521-9eeb-4bfc-917f-185903823c43-kube-api-access-9jxnz" (OuterVolumeSpecName: "kube-api-access-9jxnz") pod "98535521-9eeb-4bfc-917f-185903823c43" (UID: "98535521-9eeb-4bfc-917f-185903823c43"). InnerVolumeSpecName "kube-api-access-9jxnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:25.211843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.211766    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1809341b-65c6-4c63-a63c-33ba1e3a63d8-kube-api-access-j2559" (OuterVolumeSpecName: "kube-api-access-j2559") pod "1809341b-65c6-4c63-a63c-33ba1e3a63d8" (UID: "1809341b-65c6-4c63-a63c-33ba1e3a63d8"). InnerVolumeSpecName "kube-api-access-j2559". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:25.212027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.212004    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e9fa0a-325e-4e37-a94c-c25726a26184-kube-api-access-6rq65" (OuterVolumeSpecName: "kube-api-access-6rq65") pod "56e9fa0a-325e-4e37-a94c-c25726a26184" (UID: "56e9fa0a-325e-4e37-a94c-c25726a26184"). InnerVolumeSpecName "kube-api-access-6rq65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:25.310694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.310598    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.310694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.310627    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2559\" (UniqueName: \"kubernetes.io/projected/1809341b-65c6-4c63-a63c-33ba1e3a63d8-kube-api-access-j2559\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.310694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.310639    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56e9fa0a-325e-4e37-a94c-c25726a26184-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.310694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.310653    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rq65\" (UniqueName: \"kubernetes.io/projected/56e9fa0a-325e-4e37-a94c-c25726a26184-kube-api-access-6rq65\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.310694 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.310668    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jxnz\" (UniqueName: \"kubernetes.io/projected/98535521-9eeb-4bfc-917f-185903823c43-kube-api-access-9jxnz\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:25.379657 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.379617    2583 generic.go:356] "Generic (PLEG): container finished" podID="98535521-9eeb-4bfc-917f-185903823c43" containerID="a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5" exitCode=0
Sep 18 22:43:25.379828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.379663    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"
Sep 18 22:43:25.379828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.379706    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx" event={"ID":"98535521-9eeb-4bfc-917f-185903823c43","Type":"ContainerDied","Data":"a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5"}
Sep 18 22:43:25.379828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.379757    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx" event={"ID":"98535521-9eeb-4bfc-917f-185903823c43","Type":"ContainerDied","Data":"fd7dc4d554a0d1dad3d9f164739056e43da167eab727611a46e70d2ba7b5ceb7"}
Sep 18 22:43:25.379828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.379783    2583 scope.go:117] "RemoveContainer" containerID="a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5"
Sep 18 22:43:25.381466 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.381442    2583 generic.go:356] "Generic (PLEG): container finished" podID="56e9fa0a-325e-4e37-a94c-c25726a26184" containerID="7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1" exitCode=0
Sep 18 22:43:25.381610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.381491    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4" event={"ID":"56e9fa0a-325e-4e37-a94c-c25726a26184","Type":"ContainerDied","Data":"7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1"}
Sep 18 22:43:25.381610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.381507    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"
Sep 18 22:43:25.381610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.381540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4" event={"ID":"56e9fa0a-325e-4e37-a94c-c25726a26184","Type":"ContainerDied","Data":"9f2be614130303cb611c78494a3182e4e86cd4b6b8232c425ccdbc2381faa848"}
Sep 18 22:43:25.383382 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.383350    2583 generic.go:356] "Generic (PLEG): container finished" podID="1809341b-65c6-4c63-a63c-33ba1e3a63d8" containerID="90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0" exitCode=0
Sep 18 22:43:25.383506 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.383381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2" event={"ID":"1809341b-65c6-4c63-a63c-33ba1e3a63d8","Type":"ContainerDied","Data":"90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0"}
Sep 18 22:43:25.383506 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.383424    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"
Sep 18 22:43:25.383506 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.383457    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2" event={"ID":"1809341b-65c6-4c63-a63c-33ba1e3a63d8","Type":"ContainerDied","Data":"1e03a1b22f48140c6d0e306ce09897307cb7a90852f81fafe583f2d8cc496715"}
Sep 18 22:43:25.389763 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.389749    2583 scope.go:117] "RemoveContainer" containerID="a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5"
Sep 18 22:43:25.390086 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:43:25.390066    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5\": container with ID starting with a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5 not found: ID does not exist" containerID="a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5"
Sep 18 22:43:25.390212 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.390094    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5"} err="failed to get container status \"a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5\": rpc error: code = NotFound desc = could not find container \"a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5\": container with ID starting with a3f5202bed7ee5ebc29d72b391d12cc6bfe9a29b2e5ec80ba299cf4a38f62ff5 not found: ID does not exist"
Sep 18 22:43:25.390212 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.390110    2583 scope.go:117] "RemoveContainer" containerID="7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1"
Sep 18 22:43:25.399197 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.399180    2583 scope.go:117] "RemoveContainer" containerID="7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1"
Sep 18 22:43:25.399502 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:43:25.399475    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1\": container with ID starting with 7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1 not found: ID does not exist" containerID="7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1"
Sep 18 22:43:25.399589 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.399527    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1"} err="failed to get container status \"7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1\": rpc error: code = NotFound desc = could not find container \"7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1\": container with ID starting with 7e31bc942b61bae22f1bb349b772b04fd68dee5e2aaa8dfb7e2cc800a1d873b1 not found: ID does not exist"
Sep 18 22:43:25.399589 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.399550    2583 scope.go:117] "RemoveContainer" containerID="90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0"
Sep 18 22:43:25.407879 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.407607    2583 scope.go:117] "RemoveContainer" containerID="90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0"
Sep 18 22:43:25.408002 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:43:25.407889    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0\": container with ID starting with 90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0 not found: ID does not exist" containerID="90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0"
Sep 18 22:43:25.408002 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.407924    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0"} err="failed to get container status \"90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0\": rpc error: code = NotFound desc = could not find container \"90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0\": container with ID starting with 90dd89eb707e8ff9f415a0a1c87c68d239ae6ed883553e39e6b7580902276bf0 not found: ID does not exist"
Sep 18 22:43:25.410097 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.410077    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"]
Sep 18 22:43:25.413443 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.413413    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8faaa1-5c8b795957-nvdfx"]
Sep 18 22:43:25.425260 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.425226    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"]
Sep 18 22:43:25.434125 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.434093    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8faaa1-67f9b88f8-cd4s2"]
Sep 18 22:43:25.445555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.445530    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"]
Sep 18 22:43:25.449870 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.449845    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8faaa1-6d5cbd88b6-fnfv4"]
Sep 18 22:43:25.683199 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.683158    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1809341b-65c6-4c63-a63c-33ba1e3a63d8" path="/var/lib/kubelet/pods/1809341b-65c6-4c63-a63c-33ba1e3a63d8/volumes"
Sep 18 22:43:25.683702 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.683681    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e9fa0a-325e-4e37-a94c-c25726a26184" path="/var/lib/kubelet/pods/56e9fa0a-325e-4e37-a94c-c25726a26184/volumes"
Sep 18 22:43:25.684162 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:25.684141    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98535521-9eeb-4bfc-917f-185903823c43" path="/var/lib/kubelet/pods/98535521-9eeb-4bfc-917f-185903823c43/volumes"
Sep 18 22:43:43.366671 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.366590    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"]
Sep 18 22:43:43.367192 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.366830    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" podUID="91be9c9e-b867-4e96-8b49-4d70309d0180" containerName="maas-api" containerID="cri-o://02afc6f0c3e98b7413d6d21bbcde6023c959f8007fea7a7ecd395996a0ae6356" gracePeriod=30
Sep 18 22:43:43.461981 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.461949    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"]
Sep 18 22:43:43.462256 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.462214    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq" podUID="3cf52bda-1bf8-4cf2-aa67-4339dc817f20" containerName="payload-pre-processing" containerID="cri-o://f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801" gracePeriod=30
Sep 18 22:43:43.496471 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.496443    2583 generic.go:356] "Generic (PLEG): container finished" podID="91be9c9e-b867-4e96-8b49-4d70309d0180" containerID="02afc6f0c3e98b7413d6d21bbcde6023c959f8007fea7a7ecd395996a0ae6356" exitCode=0
Sep 18 22:43:43.496606 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.496535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" event={"ID":"91be9c9e-b867-4e96-8b49-4d70309d0180","Type":"ContainerDied","Data":"02afc6f0c3e98b7413d6d21bbcde6023c959f8007fea7a7ecd395996a0ae6356"}
Sep 18 22:43:43.642198 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.642174    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:43:43.692163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.692117    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls\") pod \"91be9c9e-b867-4e96-8b49-4d70309d0180\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") "
Sep 18 22:43:43.692347 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.692235    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-metrics-certs\") pod \"91be9c9e-b867-4e96-8b49-4d70309d0180\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") "
Sep 18 22:43:43.692710 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.692629    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91be9c9e-b867-4e96-8b49-4d70309d0180-openshift-service-ca\") pod \"91be9c9e-b867-4e96-8b49-4d70309d0180\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") "
Sep 18 22:43:43.692827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.692772    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb9p8\" (UniqueName: \"kubernetes.io/projected/91be9c9e-b867-4e96-8b49-4d70309d0180-kube-api-access-bb9p8\") pod \"91be9c9e-b867-4e96-8b49-4d70309d0180\" (UID: \"91be9c9e-b867-4e96-8b49-4d70309d0180\") "
Sep 18 22:43:43.692827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.692808    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91be9c9e-b867-4e96-8b49-4d70309d0180-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91be9c9e-b867-4e96-8b49-4d70309d0180" (UID: "91be9c9e-b867-4e96-8b49-4d70309d0180"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:43.693481 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.693161    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91be9c9e-b867-4e96-8b49-4d70309d0180-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:43.695419 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.695369    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "91be9c9e-b867-4e96-8b49-4d70309d0180" (UID: "91be9c9e-b867-4e96-8b49-4d70309d0180"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:43:43.696724 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.696693    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91be9c9e-b867-4e96-8b49-4d70309d0180" (UID: "91be9c9e-b867-4e96-8b49-4d70309d0180"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:43:43.696832 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.696771    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91be9c9e-b867-4e96-8b49-4d70309d0180-kube-api-access-bb9p8" (OuterVolumeSpecName: "kube-api-access-bb9p8") pod "91be9c9e-b867-4e96-8b49-4d70309d0180" (UID: "91be9c9e-b867-4e96-8b49-4d70309d0180"). InnerVolumeSpecName "kube-api-access-bb9p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:43.726355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.726331    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:43:43.794369 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.794336    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-plugins-config-volume\") pod \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") "
Sep 18 22:43:43.794588 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.794445    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krqwc\" (UniqueName: \"kubernetes.io/projected/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-kube-api-access-krqwc\") pod \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\" (UID: \"3cf52bda-1bf8-4cf2-aa67-4339dc817f20\") "
Sep 18 22:43:43.794717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.794689    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cf52bda-1bf8-4cf2-aa67-4339dc817f20" (UID: "3cf52bda-1bf8-4cf2-aa67-4339dc817f20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:43.794777 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.794765    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:43.794829 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.794785    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91be9c9e-b867-4e96-8b49-4d70309d0180-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:43.794829 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.794799    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb9p8\" (UniqueName: \"kubernetes.io/projected/91be9c9e-b867-4e96-8b49-4d70309d0180-kube-api-access-bb9p8\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:43.796478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.796458    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-kube-api-access-krqwc" (OuterVolumeSpecName: "kube-api-access-krqwc") pod "3cf52bda-1bf8-4cf2-aa67-4339dc817f20" (UID: "3cf52bda-1bf8-4cf2-aa67-4339dc817f20"). InnerVolumeSpecName "kube-api-access-krqwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:43.895722 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.895690    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:43.895722 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:43.895722    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krqwc\" (UniqueName: \"kubernetes.io/projected/3cf52bda-1bf8-4cf2-aa67-4339dc817f20-kube-api-access-krqwc\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:44.503964 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.503933    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"
Sep 18 22:43:44.503964 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.503947    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx" event={"ID":"91be9c9e-b867-4e96-8b49-4d70309d0180","Type":"ContainerDied","Data":"2e3706c091da8f6d3ef39e268f47f195edbbb5fb2cc8b6363bc190d36f96008a"}
Sep 18 22:43:44.504501 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.504042    2583 scope.go:117] "RemoveContainer" containerID="02afc6f0c3e98b7413d6d21bbcde6023c959f8007fea7a7ecd395996a0ae6356"
Sep 18 22:43:44.505644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.505528    2583 generic.go:356] "Generic (PLEG): container finished" podID="3cf52bda-1bf8-4cf2-aa67-4339dc817f20" containerID="f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801" exitCode=0
Sep 18 22:43:44.505644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.505583    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"
Sep 18 22:43:44.505778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.505584    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq" event={"ID":"3cf52bda-1bf8-4cf2-aa67-4339dc817f20","Type":"ContainerDied","Data":"f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801"}
Sep 18 22:43:44.505778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.505680    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq" event={"ID":"3cf52bda-1bf8-4cf2-aa67-4339dc817f20","Type":"ContainerDied","Data":"7aa45f4e68e84057951059e01674cde0e0e3d0116eca4e72a82d43c4d996ae6c"}
Sep 18 22:43:44.514946 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.514927    2583 scope.go:117] "RemoveContainer" containerID="f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801"
Sep 18 22:43:44.523525 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.523503    2583 scope.go:117] "RemoveContainer" containerID="f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801"
Sep 18 22:43:44.523861 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:43:44.523838    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801\": container with ID starting with f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801 not found: ID does not exist" containerID="f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801"
Sep 18 22:43:44.523964 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.523869    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801"} err="failed to get container status \"f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801\": rpc error: code = NotFound desc = could not find container \"f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801\": container with ID starting with f6ca81d259d832d5fce01d7c0d0cb3e47d8eb4b5e8c41acd900cab950bfd5801 not found: ID does not exist"
Sep 18 22:43:44.529096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.529070    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"]
Sep 18 22:43:44.535149 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.535127    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-20531f-78859cc6c6-t6ccx"]
Sep 18 22:43:44.545849 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.545826    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"]
Sep 18 22:43:44.549925 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:44.549901    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-20531f-776868bd6-9zvzq"]
Sep 18 22:43:45.683367 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:45.683336    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf52bda-1bf8-4cf2-aa67-4339dc817f20" path="/var/lib/kubelet/pods/3cf52bda-1bf8-4cf2-aa67-4339dc817f20/volumes"
Sep 18 22:43:45.683831 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:45.683815    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91be9c9e-b867-4e96-8b49-4d70309d0180" path="/var/lib/kubelet/pods/91be9c9e-b867-4e96-8b49-4d70309d0180/volumes"
Sep 18 22:43:54.438868 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.438834    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"]
Sep 18 22:43:54.439303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.439042    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg" podUID="64c838bb-54de-491c-9a76-e7989c79f563" containerName="payload-processing" containerID="cri-o://1b1f830ab6c0c562303a6e1cf5a7098dab5bf6019ebe8cf6c206fa828950cd46" gracePeriod=30
Sep 18 22:43:54.452920 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.452891    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"]
Sep 18 22:43:54.453187 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.453138    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb" podUID="61d2b728-a8ff-4aea-8a07-52de0cb1e77e" containerName="payload-pre-processing" containerID="cri-o://bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea" gracePeriod=30
Sep 18 22:43:54.571179 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.571139    2583 generic.go:356] "Generic (PLEG): container finished" podID="64c838bb-54de-491c-9a76-e7989c79f563" containerID="1b1f830ab6c0c562303a6e1cf5a7098dab5bf6019ebe8cf6c206fa828950cd46" exitCode=0
Sep 18 22:43:54.571573 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.571209    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg" event={"ID":"64c838bb-54de-491c-9a76-e7989c79f563","Type":"ContainerDied","Data":"1b1f830ab6c0c562303a6e1cf5a7098dab5bf6019ebe8cf6c206fa828950cd46"}
Sep 18 22:43:54.736990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.736967    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:43:54.739930 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.739904    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:43:54.794990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.794957    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-plugins-config-volume\") pod \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") "
Sep 18 22:43:54.795160 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795007    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c838bb-54de-491c-9a76-e7989c79f563-plugins-config-volume\") pod \"64c838bb-54de-491c-9a76-e7989c79f563\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") "
Sep 18 22:43:54.795160 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795112    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltnn4\" (UniqueName: \"kubernetes.io/projected/64c838bb-54de-491c-9a76-e7989c79f563-kube-api-access-ltnn4\") pod \"64c838bb-54de-491c-9a76-e7989c79f563\" (UID: \"64c838bb-54de-491c-9a76-e7989c79f563\") "
Sep 18 22:43:54.795276 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795199    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r88mm\" (UniqueName: \"kubernetes.io/projected/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-kube-api-access-r88mm\") pod \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\" (UID: \"61d2b728-a8ff-4aea-8a07-52de0cb1e77e\") "
Sep 18 22:43:54.795405 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795358    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c838bb-54de-491c-9a76-e7989c79f563-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64c838bb-54de-491c-9a76-e7989c79f563" (UID: "64c838bb-54de-491c-9a76-e7989c79f563"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:54.795494 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795407    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61d2b728-a8ff-4aea-8a07-52de0cb1e77e" (UID: "61d2b728-a8ff-4aea-8a07-52de0cb1e77e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:43:54.795552 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795539    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:54.795593 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.795559    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64c838bb-54de-491c-9a76-e7989c79f563-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:54.798021 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.797995    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-kube-api-access-r88mm" (OuterVolumeSpecName: "kube-api-access-r88mm") pod "61d2b728-a8ff-4aea-8a07-52de0cb1e77e" (UID: "61d2b728-a8ff-4aea-8a07-52de0cb1e77e"). InnerVolumeSpecName "kube-api-access-r88mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:54.798146 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.798027    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c838bb-54de-491c-9a76-e7989c79f563-kube-api-access-ltnn4" (OuterVolumeSpecName: "kube-api-access-ltnn4") pod "64c838bb-54de-491c-9a76-e7989c79f563" (UID: "64c838bb-54de-491c-9a76-e7989c79f563"). InnerVolumeSpecName "kube-api-access-ltnn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:43:54.896371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.896333    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r88mm\" (UniqueName: \"kubernetes.io/projected/61d2b728-a8ff-4aea-8a07-52de0cb1e77e-kube-api-access-r88mm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:54.896371 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:54.896363    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltnn4\" (UniqueName: \"kubernetes.io/projected/64c838bb-54de-491c-9a76-e7989c79f563-kube-api-access-ltnn4\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:43:55.578115 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.578081    2583 generic.go:356] "Generic (PLEG): container finished" podID="61d2b728-a8ff-4aea-8a07-52de0cb1e77e" containerID="bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea" exitCode=0
Sep 18 22:43:55.578557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.578130    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"
Sep 18 22:43:55.578557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.578172    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb" event={"ID":"61d2b728-a8ff-4aea-8a07-52de0cb1e77e","Type":"ContainerDied","Data":"bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea"}
Sep 18 22:43:55.578557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.578227    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb" event={"ID":"61d2b728-a8ff-4aea-8a07-52de0cb1e77e","Type":"ContainerDied","Data":"2e6a0aeb2971436d18f5f09af8b94d554d35cdd1dd686f5907952bff3bceee44"}
Sep 18 22:43:55.578557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.578250    2583 scope.go:117] "RemoveContainer" containerID="bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea"
Sep 18 22:43:55.579705 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.579640    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"
Sep 18 22:43:55.579705 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.579669    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg" event={"ID":"64c838bb-54de-491c-9a76-e7989c79f563","Type":"ContainerDied","Data":"24204724c8f668a1ce9b8bea750b2bb3c1035fdad7a1c6e16681e24af40b35c7"}
Sep 18 22:43:55.588163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.588148    2583 scope.go:117] "RemoveContainer" containerID="bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea"
Sep 18 22:43:55.588522 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:43:55.588498    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea\": container with ID starting with bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea not found: ID does not exist" containerID="bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea"
Sep 18 22:43:55.588591 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.588532    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea"} err="failed to get container status \"bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea\": rpc error: code = NotFound desc = could not find container \"bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea\": container with ID starting with bd1d3b907c0411bddc01677220129b2ee666854c29f57a542818ff2dbc7a4cea not found: ID does not exist"
Sep 18 22:43:55.588591 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.588550    2583 scope.go:117] "RemoveContainer" containerID="1b1f830ab6c0c562303a6e1cf5a7098dab5bf6019ebe8cf6c206fa828950cd46"
Sep 18 22:43:55.622662 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.622632    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"]
Sep 18 22:43:55.630518 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.630494    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f9cf33-86d45b94d7-2cksg"]
Sep 18 22:43:55.660173 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.660141    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"]
Sep 18 22:43:55.683017 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.682984    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c838bb-54de-491c-9a76-e7989c79f563" path="/var/lib/kubelet/pods/64c838bb-54de-491c-9a76-e7989c79f563/volumes"
Sep 18 22:43:55.690797 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:55.690775    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f9cf33-6f76c48d58-fvkpb"]
Sep 18 22:43:57.684644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:43:57.684608    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61d2b728-a8ff-4aea-8a07-52de0cb1e77e" path="/var/lib/kubelet/pods/61d2b728-a8ff-4aea-8a07-52de0cb1e77e/volumes"
Sep 18 22:44:14.243581 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:14.243549    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"]
Sep 18 22:44:29.262110 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.262039    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7" podUID="731d4315-b4e2-4b01-9f83-0a5433616ed7" containerName="main" containerID="cri-o://235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222" gracePeriod=45
Sep 18 22:44:29.518210 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.518153    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:44:29.607022 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.606988    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-tmp-dir\") pod \"731d4315-b4e2-4b01-9f83-0a5433616ed7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") "
Sep 18 22:44:29.607187 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607048    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-model-cache\") pod \"731d4315-b4e2-4b01-9f83-0a5433616ed7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") "
Sep 18 22:44:29.607187 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607101    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731d4315-b4e2-4b01-9f83-0a5433616ed7-tls-certs\") pod \"731d4315-b4e2-4b01-9f83-0a5433616ed7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") "
Sep 18 22:44:29.607187 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607157    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-home\") pod \"731d4315-b4e2-4b01-9f83-0a5433616ed7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") "
Sep 18 22:44:29.607334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607187    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-dshm\") pod \"731d4315-b4e2-4b01-9f83-0a5433616ed7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") "
Sep 18 22:44:29.607334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607228    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tq7s\" (UniqueName: \"kubernetes.io/projected/731d4315-b4e2-4b01-9f83-0a5433616ed7-kube-api-access-5tq7s\") pod \"731d4315-b4e2-4b01-9f83-0a5433616ed7\" (UID: \"731d4315-b4e2-4b01-9f83-0a5433616ed7\") "
Sep 18 22:44:29.607484 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607345    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "731d4315-b4e2-4b01-9f83-0a5433616ed7" (UID: "731d4315-b4e2-4b01-9f83-0a5433616ed7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:44:29.607484 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607356    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-model-cache" (OuterVolumeSpecName: "model-cache") pod "731d4315-b4e2-4b01-9f83-0a5433616ed7" (UID: "731d4315-b4e2-4b01-9f83-0a5433616ed7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:44:29.607484 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607439    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-home" (OuterVolumeSpecName: "home") pod "731d4315-b4e2-4b01-9f83-0a5433616ed7" (UID: "731d4315-b4e2-4b01-9f83-0a5433616ed7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:44:29.607636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607576    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:44:29.607636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607590    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:44:29.607636 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.607603    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:44:29.609531 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.609505    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-dshm" (OuterVolumeSpecName: "dshm") pod "731d4315-b4e2-4b01-9f83-0a5433616ed7" (UID: "731d4315-b4e2-4b01-9f83-0a5433616ed7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 22:44:29.609618 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.609560    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731d4315-b4e2-4b01-9f83-0a5433616ed7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "731d4315-b4e2-4b01-9f83-0a5433616ed7" (UID: "731d4315-b4e2-4b01-9f83-0a5433616ed7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:44:29.609661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.609637    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731d4315-b4e2-4b01-9f83-0a5433616ed7-kube-api-access-5tq7s" (OuterVolumeSpecName: "kube-api-access-5tq7s") pod "731d4315-b4e2-4b01-9f83-0a5433616ed7" (UID: "731d4315-b4e2-4b01-9f83-0a5433616ed7"). InnerVolumeSpecName "kube-api-access-5tq7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:44:29.708864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.708829    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731d4315-b4e2-4b01-9f83-0a5433616ed7-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:44:29.708864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.708860    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731d4315-b4e2-4b01-9f83-0a5433616ed7-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:44:29.709109 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.708872    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tq7s\" (UniqueName: \"kubernetes.io/projected/731d4315-b4e2-4b01-9f83-0a5433616ed7-kube-api-access-5tq7s\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:44:29.764332 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.764293    2583 generic.go:356] "Generic (PLEG): container finished" podID="731d4315-b4e2-4b01-9f83-0a5433616ed7" containerID="235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222" exitCode=0
Sep 18 22:44:29.764529 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.764349    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7" event={"ID":"731d4315-b4e2-4b01-9f83-0a5433616ed7","Type":"ContainerDied","Data":"235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222"}
Sep 18 22:44:29.764529 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.764358    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"
Sep 18 22:44:29.764529 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.764411    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7" event={"ID":"731d4315-b4e2-4b01-9f83-0a5433616ed7","Type":"ContainerDied","Data":"b79376cdd30ff698dadbc024e0026eaa38a7acdf698fb33ee0af16695ab3fcb0"}
Sep 18 22:44:29.764529 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.764436    2583 scope.go:117] "RemoveContainer" containerID="235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222"
Sep 18 22:44:29.773154 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.773137    2583 scope.go:117] "RemoveContainer" containerID="235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222"
Sep 18 22:44:29.773429 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:44:29.773409    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222\": container with ID starting with 235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222 not found: ID does not exist" containerID="235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222"
Sep 18 22:44:29.773498 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.773437    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222"} err="failed to get container status \"235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222\": rpc error: code = NotFound desc = could not find container \"235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222\": container with ID starting with 235220c3f0edb7f401fd91c87862348da1f84e9d81c134208e39bd060d924222 not found: ID does not exist"
Sep 18 22:44:29.781745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.781723    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"]
Sep 18 22:44:29.785987 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:29.785965    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a91c21/auth-test-model-a91c21-kserve-678bcb865f-7r5p7"]
Sep 18 22:44:31.683399 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:44:31.683362    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="731d4315-b4e2-4b01-9f83-0a5433616ed7" path="/var/lib/kubelet/pods/731d4315-b4e2-4b01-9f83-0a5433616ed7/volumes"
Sep 18 22:45:00.143762 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.143721    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"]
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144085    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c838bb-54de-491c-9a76-e7989c79f563" containerName="payload-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144096    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c838bb-54de-491c-9a76-e7989c79f563" containerName="payload-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144107    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf52bda-1bf8-4cf2-aa67-4339dc817f20" containerName="payload-pre-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144113    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf52bda-1bf8-4cf2-aa67-4339dc817f20" containerName="payload-pre-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144122    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1809341b-65c6-4c63-a63c-33ba1e3a63d8" containerName="payload-pre-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144128    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1809341b-65c6-4c63-a63c-33ba1e3a63d8" containerName="payload-pre-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144137    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731d4315-b4e2-4b01-9f83-0a5433616ed7" containerName="main"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144143    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="731d4315-b4e2-4b01-9f83-0a5433616ed7" containerName="main"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144149    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" containerName="maas-api"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144155    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" containerName="maas-api"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144162    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e9fa0a-325e-4e37-a94c-c25726a26184" containerName="maas-api"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144167    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e9fa0a-325e-4e37-a94c-c25726a26184" containerName="maas-api"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144174    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ac49ea-4f3b-477f-8fb3-403d5ae45fae" containerName="payload-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144179    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ac49ea-4f3b-477f-8fb3-403d5ae45fae" containerName="payload-processing"
Sep 18 22:45:00.144180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144187    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d9dd7a-6198-4228-b827-35482130c7b4" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144192    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d9dd7a-6198-4228-b827-35482130c7b4" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144200    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91be9c9e-b867-4e96-8b49-4d70309d0180" containerName="maas-api"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144206    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="91be9c9e-b867-4e96-8b49-4d70309d0180" containerName="maas-api"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144214    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98535521-9eeb-4bfc-917f-185903823c43" containerName="payload-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144220    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="98535521-9eeb-4bfc-917f-185903823c43" containerName="payload-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144230    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61d2b728-a8ff-4aea-8a07-52de0cb1e77e" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144235    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d2b728-a8ff-4aea-8a07-52de0cb1e77e" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144240    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144245    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144251    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144256    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144305    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c838bb-54de-491c-9a76-e7989c79f563" containerName="payload-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144313    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="91be9c9e-b867-4e96-8b49-4d70309d0180" containerName="maas-api"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144318    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="731d4315-b4e2-4b01-9f83-0a5433616ed7" containerName="main"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144323    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ac49ea-4f3b-477f-8fb3-403d5ae45fae" containerName="payload-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144329    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d9dd7a-6198-4228-b827-35482130c7b4" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144335    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="61d2b728-a8ff-4aea-8a07-52de0cb1e77e" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144340    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1809341b-65c6-4c63-a63c-33ba1e3a63d8" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144346    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e9fa0a-325e-4e37-a94c-c25726a26184" containerName="maas-api"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144351    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9640f3e-1a57-4535-aa72-ffe1fc71cfbe" containerName="maas-api"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144355    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf52bda-1bf8-4cf2-aa67-4339dc817f20" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144360    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ba5ac0-d0f4-47e7-8a54-ff1f6a53e427" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144366    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38f1bd2-99b8-4cd8-bbdf-8dcc3591e2b0" containerName="payload-pre-processing"
Sep 18 22:45:00.144661 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.144370    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="98535521-9eeb-4bfc-917f-185903823c43" containerName="payload-processing"
Sep 18 22:45:00.148500 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.148485    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"
Sep 18 22:45:00.151323 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.151298    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8dgjh\""
Sep 18 22:45:00.155336 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.155313    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"]
Sep 18 22:45:00.161822 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.161795    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"
Sep 18 22:45:00.434787 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.434711    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"]
Sep 18 22:45:00.437790 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:45:00.437758    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f247812_7caf_4b55_b99d_a152dc737f24.slice/crio-8e0e03de75162189f9bba9e5d5f495693d542b5741d8d86e60231be381f57031 WatchSource:0}: Error finding container 8e0e03de75162189f9bba9e5d5f495693d542b5741d8d86e60231be381f57031: Status 404 returned error can't find the container with id 8e0e03de75162189f9bba9e5d5f495693d542b5741d8d86e60231be381f57031
Sep 18 22:45:00.936297 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.936261    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh" event={"ID":"5f247812-7caf-4b55-b99d-a152dc737f24","Type":"ContainerStarted","Data":"c6d0bbdaa735265141831c2e8134f90007a4e175400da8f0913c76c655ba84f7"}
Sep 18 22:45:00.936535 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.936314    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh" event={"ID":"5f247812-7caf-4b55-b99d-a152dc737f24","Type":"ContainerStarted","Data":"8e0e03de75162189f9bba9e5d5f495693d542b5741d8d86e60231be381f57031"}
Sep 18 22:45:00.953304 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:00.953255    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh" podStartSLOduration=0.953237935 podStartE2EDuration="953.237935ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:45:00.950770385 +0000 UTC m=+1685.311287258" watchObservedRunningTime="2026-09-18 22:45:00.953237935 +0000 UTC m=+1685.313754791"
Sep 18 22:45:01.943081 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:01.943046    2583 generic.go:356] "Generic (PLEG): container finished" podID="5f247812-7caf-4b55-b99d-a152dc737f24" containerID="c6d0bbdaa735265141831c2e8134f90007a4e175400da8f0913c76c655ba84f7" exitCode=0
Sep 18 22:45:01.943486 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:01.943126    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh" event={"ID":"5f247812-7caf-4b55-b99d-a152dc737f24","Type":"ContainerDied","Data":"c6d0bbdaa735265141831c2e8134f90007a4e175400da8f0913c76c655ba84f7"}
Sep 18 22:45:03.080475 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:03.080451    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"
Sep 18 22:45:03.955029 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:03.954944    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"
Sep 18 22:45:03.955029 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:03.954981    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh" event={"ID":"5f247812-7caf-4b55-b99d-a152dc737f24","Type":"ContainerDied","Data":"8e0e03de75162189f9bba9e5d5f495693d542b5741d8d86e60231be381f57031"}
Sep 18 22:45:03.955222 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:45:03.955032    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e0e03de75162189f9bba9e5d5f495693d542b5741d8d86e60231be381f57031"
Sep 18 22:46:51.076656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.076621    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"]
Sep 18 22:46:51.077344 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.077320    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f247812-7caf-4b55-b99d-a152dc737f24" containerName="cleanup"
Sep 18 22:46:51.077344 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.077346    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f247812-7caf-4b55-b99d-a152dc737f24" containerName="cleanup"
Sep 18 22:46:51.077542 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.077482    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f247812-7caf-4b55-b99d-a152dc737f24" containerName="cleanup"
Sep 18 22:46:51.080594 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.080573    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.083144 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.083124    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-e1f79c-serving-cert\""
Sep 18 22:46:51.094479 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.094455    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"]
Sep 18 22:46:51.154051 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.154023    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39fa293-85fc-4979-9ad6-e0da0c645e26-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.154228 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.154090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.154228 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.154153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsqmg\" (UniqueName: \"kubernetes.io/projected/b39fa293-85fc-4979-9ad6-e0da0c645e26-kube-api-access-dsqmg\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.154228 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.154201    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-metrics-certs\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.255420 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.255354    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39fa293-85fc-4979-9ad6-e0da0c645e26-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.255625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.255484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.255625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.255522    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsqmg\" (UniqueName: \"kubernetes.io/projected/b39fa293-85fc-4979-9ad6-e0da0c645e26-kube-api-access-dsqmg\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.255625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.255564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-metrics-certs\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.255805 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:46:51.255630    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-serving-cert: secret "maas-api-e2e-ipp-a-e1f79c-serving-cert" not found
Sep 18 22:46:51.255805 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:46:51.255731    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls podName:b39fa293-85fc-4979-9ad6-e0da0c645e26 nodeName:}" failed. No retries permitted until 2026-09-18 22:46:51.755716047 +0000 UTC m=+1796.116232881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls") pod "maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" (UID: "b39fa293-85fc-4979-9ad6-e0da0c645e26") : secret "maas-api-e2e-ipp-a-e1f79c-serving-cert" not found
Sep 18 22:46:51.256084 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.256063    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39fa293-85fc-4979-9ad6-e0da0c645e26-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.258024 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.258003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-metrics-certs\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.268687 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.268667    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsqmg\" (UniqueName: \"kubernetes.io/projected/b39fa293-85fc-4979-9ad6-e0da0c645e26-kube-api-access-dsqmg\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.760430 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.760364    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.763186 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.763149    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:51.839037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.838995    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"]
Sep 18 22:46:51.842719 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.842701    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:51.850757 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.850738    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e1f79c-dockercfg-d4f58\""
Sep 18 22:46:51.851018 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.851001    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e1f79c\""
Sep 18 22:46:51.870462 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.870438    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"]
Sep 18 22:46:51.908456 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.908422    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"]
Sep 18 22:46:51.911968 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.911948    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:51.949806 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.949769    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"]
Sep 18 22:46:51.963557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.963524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58kx5\" (UniqueName: \"kubernetes.io/projected/dcb7ab72-26a2-4a70-bbca-d2204c00a845-kube-api-access-58kx5\") pod \"payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:51.963711 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.963608    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb7ab72-26a2-4a70-bbca-d2204c00a845-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:51.991588 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:51.991555    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:52.064944 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.064910    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhsvs\" (UniqueName: \"kubernetes.io/projected/784e159c-a2b7-4659-9a1c-dc00a0741516-kube-api-access-qhsvs\") pod \"payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.065113 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.064971    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784e159c-a2b7-4659-9a1c-dc00a0741516-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.065113 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.065083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58kx5\" (UniqueName: \"kubernetes.io/projected/dcb7ab72-26a2-4a70-bbca-d2204c00a845-kube-api-access-58kx5\") pod \"payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:52.065237 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.065152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb7ab72-26a2-4a70-bbca-d2204c00a845-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:52.065848 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.065821    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb7ab72-26a2-4a70-bbca-d2204c00a845-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:52.078644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.078620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58kx5\" (UniqueName: \"kubernetes.io/projected/dcb7ab72-26a2-4a70-bbca-d2204c00a845-kube-api-access-58kx5\") pod \"payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:52.153052 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.153019    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:52.166440 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.166411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhsvs\" (UniqueName: \"kubernetes.io/projected/784e159c-a2b7-4659-9a1c-dc00a0741516-kube-api-access-qhsvs\") pod \"payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.166512 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.166472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784e159c-a2b7-4659-9a1c-dc00a0741516-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.167166 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.167146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784e159c-a2b7-4659-9a1c-dc00a0741516-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.176497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.176467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhsvs\" (UniqueName: \"kubernetes.io/projected/784e159c-a2b7-4659-9a1c-dc00a0741516-kube-api-access-qhsvs\") pod \"payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.222843 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.222812    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:52.271207 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.271172    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"]
Sep 18 22:46:52.276234 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:46:52.276102    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39fa293_85fc_4979_9ad6_e0da0c645e26.slice/crio-4e85d1ae0cc80dd3d21f786665d68d8558e40d0d8583cee0d4fcb96fc0640395 WatchSource:0}: Error finding container 4e85d1ae0cc80dd3d21f786665d68d8558e40d0d8583cee0d4fcb96fc0640395: Status 404 returned error can't find the container with id 4e85d1ae0cc80dd3d21f786665d68d8558e40d0d8583cee0d4fcb96fc0640395
Sep 18 22:46:52.429738 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.429710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"]
Sep 18 22:46:52.431789 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:46:52.431747    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb7ab72_26a2_4a70_bbca_d2204c00a845.slice/crio-58f0334309f802f8a5eee1b65fd61c3d8c92f2b63364c51c191fc1a77a924347 WatchSource:0}: Error finding container 58f0334309f802f8a5eee1b65fd61c3d8c92f2b63364c51c191fc1a77a924347: Status 404 returned error can't find the container with id 58f0334309f802f8a5eee1b65fd61c3d8c92f2b63364c51c191fc1a77a924347
Sep 18 22:46:52.501757 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.501729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"]
Sep 18 22:46:52.503608 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:46:52.503585    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784e159c_a2b7_4659_9a1c_dc00a0741516.slice/crio-dce9ea5f56a9f91d1ddd87a85ca486f4d4ba98c533de12bd9f01e13011751847 WatchSource:0}: Error finding container dce9ea5f56a9f91d1ddd87a85ca486f4d4ba98c533de12bd9f01e13011751847: Status 404 returned error can't find the container with id dce9ea5f56a9f91d1ddd87a85ca486f4d4ba98c533de12bd9f01e13011751847
Sep 18 22:46:52.531043 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.531007    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb" event={"ID":"784e159c-a2b7-4659-9a1c-dc00a0741516","Type":"ContainerStarted","Data":"dce9ea5f56a9f91d1ddd87a85ca486f4d4ba98c533de12bd9f01e13011751847"}
Sep 18 22:46:52.532868 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.532841    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8" event={"ID":"dcb7ab72-26a2-4a70-bbca-d2204c00a845","Type":"ContainerStarted","Data":"34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67"}
Sep 18 22:46:52.533037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.532890    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8" event={"ID":"dcb7ab72-26a2-4a70-bbca-d2204c00a845","Type":"ContainerStarted","Data":"58f0334309f802f8a5eee1b65fd61c3d8c92f2b63364c51c191fc1a77a924347"}
Sep 18 22:46:52.533037 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.533013    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:46:52.534341 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.534317    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" event={"ID":"b39fa293-85fc-4979-9ad6-e0da0c645e26","Type":"ContainerStarted","Data":"4e85d1ae0cc80dd3d21f786665d68d8558e40d0d8583cee0d4fcb96fc0640395"}
Sep 18 22:46:52.551672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:52.551616    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8" podStartSLOduration=1.551594752 podStartE2EDuration="1.551594752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:46:52.549838265 +0000 UTC m=+1796.910355122" watchObservedRunningTime="2026-09-18 22:46:52.551594752 +0000 UTC m=+1796.912111612"
Sep 18 22:46:53.541733 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:53.541695    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" event={"ID":"b39fa293-85fc-4979-9ad6-e0da0c645e26","Type":"ContainerStarted","Data":"ff984756395e84e77e6821e1ab490f61fc91a5b983b1c97191f1aec863422246"}
Sep 18 22:46:53.542209 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:53.541861    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:46:53.543304 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:53.543279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb" event={"ID":"784e159c-a2b7-4659-9a1c-dc00a0741516","Type":"ContainerStarted","Data":"2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db"}
Sep 18 22:46:53.575516 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:53.575462    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" podStartSLOduration=2.119207237 podStartE2EDuration="2.575443948s" totalImagesPullingTime="456.236711ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:46:53.574076361 +0000 UTC m=+1797.934593219" watchObservedRunningTime="2026-09-18 22:46:53.575443948 +0000 UTC m=+1797.935960825"
Sep 18 22:46:53.601511 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:53.601453    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb" podStartSLOduration=2.601436337 podStartE2EDuration="2.601436337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:46:53.599073369 +0000 UTC m=+1797.959590238" watchObservedRunningTime="2026-09-18 22:46:53.601436337 +0000 UTC m=+1797.961953241"
Sep 18 22:46:54.548912 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:54.548877    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:46:57.751612 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:57.751583    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:46:57.755829 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:57.755803    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:46:59.554859 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:46:59.554829    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:47:03.546053 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:47:03.546024    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:47:05.557811 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:47:05.557769    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:47:53.703099 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:47:53.703073    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-twfmq_80653214-eddf-4e73-882c-4348f6613d7e/payload-processing/0.log"
Sep 18 22:48:24.674887 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:24.674784    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8_dcb7ab72-26a2-4a70-bbca-d2204c00a845/payload-processing/0.log"
Sep 18 22:48:24.797585 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:24.797556    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-twfmq_80653214-eddf-4e73-882c-4348f6613d7e/payload-processing/0.log"
Sep 18 22:48:32.549326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.549284    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"]
Sep 18 22:48:32.553339 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.553314    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.559882 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.559854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f696b1-serving-cert\""
Sep 18 22:48:32.568916 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.568891    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"]
Sep 18 22:48:32.589403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.589356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9674939a-dc7c-4eba-836b-f370f435fb91-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.589403 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.589404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdrwr\" (UniqueName: \"kubernetes.io/projected/9674939a-dc7c-4eba-836b-f370f435fb91-kube-api-access-mdrwr\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.589601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.589503    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.589601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.589570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.690815 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.690780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9674939a-dc7c-4eba-836b-f370f435fb91-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.690815 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.690820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdrwr\" (UniqueName: \"kubernetes.io/projected/9674939a-dc7c-4eba-836b-f370f435fb91-kube-api-access-mdrwr\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.691100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.690868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.691100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.690901    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.691536 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.691512    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9674939a-dc7c-4eba-836b-f370f435fb91-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.693501 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.693472    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.693605 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.693472    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.703255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.703226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdrwr\" (UniqueName: \"kubernetes.io/projected/9674939a-dc7c-4eba-836b-f370f435fb91-kube-api-access-mdrwr\") pod \"maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:32.864447 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:32.864329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:33.147252 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.147222    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"]
Sep 18 22:48:33.148354 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:48:33.148316    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9674939a_dc7c_4eba_836b_f370f435fb91.slice/crio-aca1b227ca135daaa21d8c1a1146a048676dc02855ae2978bcf14aacd8c68e18 WatchSource:0}: Error finding container aca1b227ca135daaa21d8c1a1146a048676dc02855ae2978bcf14aacd8c68e18: Status 404 returned error can't find the container with id aca1b227ca135daaa21d8c1a1146a048676dc02855ae2978bcf14aacd8c68e18
Sep 18 22:48:33.150327 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.150306    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:48:33.265749 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.265713    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"]
Sep 18 22:48:33.270231 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.270215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.272660 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.272625    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-f696b1-dockercfg-8bgrb\""
Sep 18 22:48:33.273353 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.273162    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-f696b1\""
Sep 18 22:48:33.295184 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.295153    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"]
Sep 18 22:48:33.296532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.296491    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb6a888d-59a3-4762-b0aa-3814027a3b5b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.296660 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.296576    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf67w\" (UniqueName: \"kubernetes.io/projected/bb6a888d-59a3-4762-b0aa-3814027a3b5b-kube-api-access-vf67w\") pod \"payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.397846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.397768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb6a888d-59a3-4762-b0aa-3814027a3b5b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.397846 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.397805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf67w\" (UniqueName: \"kubernetes.io/projected/bb6a888d-59a3-4762-b0aa-3814027a3b5b-kube-api-access-vf67w\") pod \"payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.398824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.398801    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb6a888d-59a3-4762-b0aa-3814027a3b5b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.423736 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.423700    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf67w\" (UniqueName: \"kubernetes.io/projected/bb6a888d-59a3-4762-b0aa-3814027a3b5b-kube-api-access-vf67w\") pod \"payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.580851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.580823    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:33.863790 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:33.863760    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"]
Sep 18 22:48:33.865037 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:48:33.865012    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb6a888d_59a3_4762_b0aa_3814027a3b5b.slice/crio-6e1f4df54451001da1c9ee3e3605ec459861520a21a98a90a96473f37d902cc2 WatchSource:0}: Error finding container 6e1f4df54451001da1c9ee3e3605ec459861520a21a98a90a96473f37d902cc2: Status 404 returned error can't find the container with id 6e1f4df54451001da1c9ee3e3605ec459861520a21a98a90a96473f37d902cc2
Sep 18 22:48:34.097340 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.097308    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv" event={"ID":"9674939a-dc7c-4eba-836b-f370f435fb91","Type":"ContainerStarted","Data":"89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7"}
Sep 18 22:48:34.097542 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.097358    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv" event={"ID":"9674939a-dc7c-4eba-836b-f370f435fb91","Type":"ContainerStarted","Data":"aca1b227ca135daaa21d8c1a1146a048676dc02855ae2978bcf14aacd8c68e18"}
Sep 18 22:48:34.097542 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.097423    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:34.099069 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.099045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw" event={"ID":"bb6a888d-59a3-4762-b0aa-3814027a3b5b","Type":"ContainerStarted","Data":"eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0"}
Sep 18 22:48:34.099190 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.099093    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:48:34.099190 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.099120    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw" event={"ID":"bb6a888d-59a3-4762-b0aa-3814027a3b5b","Type":"ContainerStarted","Data":"6e1f4df54451001da1c9ee3e3605ec459861520a21a98a90a96473f37d902cc2"}
Sep 18 22:48:34.122119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.122024    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv" podStartSLOduration=1.709335616 podStartE2EDuration="2.122005362s" totalImagesPullingTime="412.669746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:48:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:48:34.121979523 +0000 UTC m=+1898.482496381" watchObservedRunningTime="2026-09-18 22:48:34.122005362 +0000 UTC m=+1898.482522219"
Sep 18 22:48:34.144219 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:34.144162    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw" podStartSLOduration=1.144141043 podStartE2EDuration="1.144141043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:48:34.14100237 +0000 UTC m=+1898.501519252" watchObservedRunningTime="2026-09-18 22:48:34.144141043 +0000 UTC m=+1898.504657902"
Sep 18 22:48:40.109986 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:40.109957    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:48:45.106935 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:48:45.106903    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:49:10.785173 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:10.785138    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"]
Sep 18 22:49:10.785798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:10.785352    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv" podUID="9674939a-dc7c-4eba-836b-f370f435fb91" containerName="maas-api" containerID="cri-o://89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7" gracePeriod=30
Sep 18 22:49:10.847183 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:10.847147    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"]
Sep 18 22:49:10.847648 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:10.847616    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw" podUID="bb6a888d-59a3-4762-b0aa-3814027a3b5b" containerName="payload-processing" containerID="cri-o://eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0" gracePeriod=30
Sep 18 22:49:11.113863 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.113829    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:49:11.147855 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.147832    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:49:11.160713 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.160687    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf67w\" (UniqueName: \"kubernetes.io/projected/bb6a888d-59a3-4762-b0aa-3814027a3b5b-kube-api-access-vf67w\") pod \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") "
Sep 18 22:49:11.160866 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.160718    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdrwr\" (UniqueName: \"kubernetes.io/projected/9674939a-dc7c-4eba-836b-f370f435fb91-kube-api-access-mdrwr\") pod \"9674939a-dc7c-4eba-836b-f370f435fb91\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") "
Sep 18 22:49:11.160866 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.160749    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9674939a-dc7c-4eba-836b-f370f435fb91-openshift-service-ca\") pod \"9674939a-dc7c-4eba-836b-f370f435fb91\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") "
Sep 18 22:49:11.160866 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.160776    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-maas-api-tls\") pod \"9674939a-dc7c-4eba-836b-f370f435fb91\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") "
Sep 18 22:49:11.160866 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.160851    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-metrics-certs\") pod \"9674939a-dc7c-4eba-836b-f370f435fb91\" (UID: \"9674939a-dc7c-4eba-836b-f370f435fb91\") "
Sep 18 22:49:11.161091 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.160877    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb6a888d-59a3-4762-b0aa-3814027a3b5b-plugins-config-volume\") pod \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\" (UID: \"bb6a888d-59a3-4762-b0aa-3814027a3b5b\") "
Sep 18 22:49:11.161155 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.161124    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9674939a-dc7c-4eba-836b-f370f435fb91-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9674939a-dc7c-4eba-836b-f370f435fb91" (UID: "9674939a-dc7c-4eba-836b-f370f435fb91"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:49:11.161247 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.161228    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9674939a-dc7c-4eba-836b-f370f435fb91-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:11.161601 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.161548    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb6a888d-59a3-4762-b0aa-3814027a3b5b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb6a888d-59a3-4762-b0aa-3814027a3b5b" (UID: "bb6a888d-59a3-4762-b0aa-3814027a3b5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:49:11.163030 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.163010    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9674939a-dc7c-4eba-836b-f370f435fb91-kube-api-access-mdrwr" (OuterVolumeSpecName: "kube-api-access-mdrwr") pod "9674939a-dc7c-4eba-836b-f370f435fb91" (UID: "9674939a-dc7c-4eba-836b-f370f435fb91"). InnerVolumeSpecName "kube-api-access-mdrwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:49:11.163119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.163035    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb6a888d-59a3-4762-b0aa-3814027a3b5b-kube-api-access-vf67w" (OuterVolumeSpecName: "kube-api-access-vf67w") pod "bb6a888d-59a3-4762-b0aa-3814027a3b5b" (UID: "bb6a888d-59a3-4762-b0aa-3814027a3b5b"). InnerVolumeSpecName "kube-api-access-vf67w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:49:11.163310 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.163277    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9674939a-dc7c-4eba-836b-f370f435fb91" (UID: "9674939a-dc7c-4eba-836b-f370f435fb91"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:49:11.163427 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.163286    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9674939a-dc7c-4eba-836b-f370f435fb91" (UID: "9674939a-dc7c-4eba-836b-f370f435fb91"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:49:11.262494 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.262458    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf67w\" (UniqueName: \"kubernetes.io/projected/bb6a888d-59a3-4762-b0aa-3814027a3b5b-kube-api-access-vf67w\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:11.262494 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.262494    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdrwr\" (UniqueName: \"kubernetes.io/projected/9674939a-dc7c-4eba-836b-f370f435fb91-kube-api-access-mdrwr\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:11.262731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.262509    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:11.262731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.262521    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9674939a-dc7c-4eba-836b-f370f435fb91-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:11.262731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.262533    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb6a888d-59a3-4762-b0aa-3814027a3b5b-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:11.308064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.307960    2583 generic.go:356] "Generic (PLEG): container finished" podID="9674939a-dc7c-4eba-836b-f370f435fb91" containerID="89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7" exitCode=0
Sep 18 22:49:11.308064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.308012    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"
Sep 18 22:49:11.308064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.308044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv" event={"ID":"9674939a-dc7c-4eba-836b-f370f435fb91","Type":"ContainerDied","Data":"89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7"}
Sep 18 22:49:11.308352 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.308110    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv" event={"ID":"9674939a-dc7c-4eba-836b-f370f435fb91","Type":"ContainerDied","Data":"aca1b227ca135daaa21d8c1a1146a048676dc02855ae2978bcf14aacd8c68e18"}
Sep 18 22:49:11.308352 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.308141    2583 scope.go:117] "RemoveContainer" containerID="89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7"
Sep 18 22:49:11.309677 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.309654    2583 generic.go:356] "Generic (PLEG): container finished" podID="bb6a888d-59a3-4762-b0aa-3814027a3b5b" containerID="eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0" exitCode=0
Sep 18 22:49:11.309799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.309697    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"
Sep 18 22:49:11.309799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.309748    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw" event={"ID":"bb6a888d-59a3-4762-b0aa-3814027a3b5b","Type":"ContainerDied","Data":"eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0"}
Sep 18 22:49:11.309799 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.309783    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw" event={"ID":"bb6a888d-59a3-4762-b0aa-3814027a3b5b","Type":"ContainerDied","Data":"6e1f4df54451001da1c9ee3e3605ec459861520a21a98a90a96473f37d902cc2"}
Sep 18 22:49:11.321199 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.321134    2583 scope.go:117] "RemoveContainer" containerID="89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7"
Sep 18 22:49:11.326596 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:49:11.326562    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7\": container with ID starting with 89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7 not found: ID does not exist" containerID="89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7"
Sep 18 22:49:11.326717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.326606    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7"} err="failed to get container status \"89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7\": rpc error: code = NotFound desc = could not find container \"89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7\": container with ID starting with 89b5952c0033e3ff02e4d80bf2c55369048190ccb3eb3d2b2697cc8a0176aaa7 not found: ID does not exist"
Sep 18 22:49:11.326717 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.326631    2583 scope.go:117] "RemoveContainer" containerID="eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0"
Sep 18 22:49:11.336621 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.336593    2583 scope.go:117] "RemoveContainer" containerID="eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0"
Sep 18 22:49:11.336937 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:49:11.336913    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0\": container with ID starting with eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0 not found: ID does not exist" containerID="eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0"
Sep 18 22:49:11.337025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.336944    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0"} err="failed to get container status \"eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0\": rpc error: code = NotFound desc = could not find container \"eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0\": container with ID starting with eb68970d6b3f0ace55f7769a151a59327da6f1dc35aabbbadb85d9363e5cb0e0 not found: ID does not exist"
Sep 18 22:49:11.341584 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.341568    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"]
Sep 18 22:49:11.345462 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.345436    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f696b1-57b8d99cb7-kg6sv"]
Sep 18 22:49:11.351568 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.351547    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"]
Sep 18 22:49:11.356629 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.356609    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f696b1-c5976ddf9-rvrsw"]
Sep 18 22:49:11.682570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.682537    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9674939a-dc7c-4eba-836b-f370f435fb91" path="/var/lib/kubelet/pods/9674939a-dc7c-4eba-836b-f370f435fb91/volumes"
Sep 18 22:49:11.682907 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:11.682894    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb6a888d-59a3-4762-b0aa-3814027a3b5b" path="/var/lib/kubelet/pods/bb6a888d-59a3-4762-b0aa-3814027a3b5b/volumes"
Sep 18 22:49:37.244210 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.244163    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"]
Sep 18 22:49:37.244777 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.244551    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" podUID="b39fa293-85fc-4979-9ad6-e0da0c645e26" containerName="maas-api" containerID="cri-o://ff984756395e84e77e6821e1ab490f61fc91a5b983b1c97191f1aec863422246" gracePeriod=30
Sep 18 22:49:37.384672 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.384604    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"]
Sep 18 22:49:37.384905 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.384876    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb" podUID="784e159c-a2b7-4659-9a1c-dc00a0741516" containerName="payload-pre-processing" containerID="cri-o://2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db" gracePeriod=30
Sep 18 22:49:37.388137 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.388108    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"]
Sep 18 22:49:37.388508 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.388434    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8" podUID="dcb7ab72-26a2-4a70-bbca-d2204c00a845" containerName="payload-processing" containerID="cri-o://34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67" gracePeriod=30
Sep 18 22:49:37.455555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.455511    2583 generic.go:356] "Generic (PLEG): container finished" podID="b39fa293-85fc-4979-9ad6-e0da0c645e26" containerID="ff984756395e84e77e6821e1ab490f61fc91a5b983b1c97191f1aec863422246" exitCode=0
Sep 18 22:49:37.455724 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.455581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" event={"ID":"b39fa293-85fc-4979-9ad6-e0da0c645e26","Type":"ContainerDied","Data":"ff984756395e84e77e6821e1ab490f61fc91a5b983b1c97191f1aec863422246"}
Sep 18 22:49:37.530608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.530577    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:49:37.602945 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.602906    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-metrics-certs\") pod \"b39fa293-85fc-4979-9ad6-e0da0c645e26\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") "
Sep 18 22:49:37.603091 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.602982    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsqmg\" (UniqueName: \"kubernetes.io/projected/b39fa293-85fc-4979-9ad6-e0da0c645e26-kube-api-access-dsqmg\") pod \"b39fa293-85fc-4979-9ad6-e0da0c645e26\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") "
Sep 18 22:49:37.603091 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.603033    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39fa293-85fc-4979-9ad6-e0da0c645e26-openshift-service-ca\") pod \"b39fa293-85fc-4979-9ad6-e0da0c645e26\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") "
Sep 18 22:49:37.603091 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.603071    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls\") pod \"b39fa293-85fc-4979-9ad6-e0da0c645e26\" (UID: \"b39fa293-85fc-4979-9ad6-e0da0c645e26\") "
Sep 18 22:49:37.604047 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.603990    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b39fa293-85fc-4979-9ad6-e0da0c645e26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b39fa293-85fc-4979-9ad6-e0da0c645e26" (UID: "b39fa293-85fc-4979-9ad6-e0da0c645e26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:49:37.606683 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.606640    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39fa293-85fc-4979-9ad6-e0da0c645e26-kube-api-access-dsqmg" (OuterVolumeSpecName: "kube-api-access-dsqmg") pod "b39fa293-85fc-4979-9ad6-e0da0c645e26" (UID: "b39fa293-85fc-4979-9ad6-e0da0c645e26"). InnerVolumeSpecName "kube-api-access-dsqmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:49:37.606683 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.606660    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b39fa293-85fc-4979-9ad6-e0da0c645e26" (UID: "b39fa293-85fc-4979-9ad6-e0da0c645e26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:49:37.606901 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.606640    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b39fa293-85fc-4979-9ad6-e0da0c645e26" (UID: "b39fa293-85fc-4979-9ad6-e0da0c645e26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:49:37.633516 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.633490    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:49:37.673731 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.673703    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:49:37.704149 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704123    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784e159c-a2b7-4659-9a1c-dc00a0741516-plugins-config-volume\") pod \"784e159c-a2b7-4659-9a1c-dc00a0741516\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") "
Sep 18 22:49:37.704303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704205    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhsvs\" (UniqueName: \"kubernetes.io/projected/784e159c-a2b7-4659-9a1c-dc00a0741516-kube-api-access-qhsvs\") pod \"784e159c-a2b7-4659-9a1c-dc00a0741516\" (UID: \"784e159c-a2b7-4659-9a1c-dc00a0741516\") "
Sep 18 22:49:37.704373 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704353    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsqmg\" (UniqueName: \"kubernetes.io/projected/b39fa293-85fc-4979-9ad6-e0da0c645e26-kube-api-access-dsqmg\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.704373 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704364    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b39fa293-85fc-4979-9ad6-e0da0c645e26-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.704373 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704373    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.704585 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704381    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b39fa293-85fc-4979-9ad6-e0da0c645e26-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.704631 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.704578    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/784e159c-a2b7-4659-9a1c-dc00a0741516-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "784e159c-a2b7-4659-9a1c-dc00a0741516" (UID: "784e159c-a2b7-4659-9a1c-dc00a0741516"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:49:37.706237 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.706215    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784e159c-a2b7-4659-9a1c-dc00a0741516-kube-api-access-qhsvs" (OuterVolumeSpecName: "kube-api-access-qhsvs") pod "784e159c-a2b7-4659-9a1c-dc00a0741516" (UID: "784e159c-a2b7-4659-9a1c-dc00a0741516"). InnerVolumeSpecName "kube-api-access-qhsvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:49:37.805641 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.805562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58kx5\" (UniqueName: \"kubernetes.io/projected/dcb7ab72-26a2-4a70-bbca-d2204c00a845-kube-api-access-58kx5\") pod \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") "
Sep 18 22:49:37.805790 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.805656    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb7ab72-26a2-4a70-bbca-d2204c00a845-plugins-config-volume\") pod \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\" (UID: \"dcb7ab72-26a2-4a70-bbca-d2204c00a845\") "
Sep 18 22:49:37.805828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.805810    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhsvs\" (UniqueName: \"kubernetes.io/projected/784e159c-a2b7-4659-9a1c-dc00a0741516-kube-api-access-qhsvs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.805828 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.805820    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784e159c-a2b7-4659-9a1c-dc00a0741516-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.806047 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.806022    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcb7ab72-26a2-4a70-bbca-d2204c00a845-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dcb7ab72-26a2-4a70-bbca-d2204c00a845" (UID: "dcb7ab72-26a2-4a70-bbca-d2204c00a845"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:49:37.807711 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.807693    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcb7ab72-26a2-4a70-bbca-d2204c00a845-kube-api-access-58kx5" (OuterVolumeSpecName: "kube-api-access-58kx5") pod "dcb7ab72-26a2-4a70-bbca-d2204c00a845" (UID: "dcb7ab72-26a2-4a70-bbca-d2204c00a845"). InnerVolumeSpecName "kube-api-access-58kx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:49:37.907150 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.907106    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58kx5\" (UniqueName: \"kubernetes.io/projected/dcb7ab72-26a2-4a70-bbca-d2204c00a845-kube-api-access-58kx5\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:37.907150 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:37.907150    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcb7ab72-26a2-4a70-bbca-d2204c00a845-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:49:38.462294 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.462261    2583 generic.go:356] "Generic (PLEG): container finished" podID="784e159c-a2b7-4659-9a1c-dc00a0741516" containerID="2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db" exitCode=0
Sep 18 22:49:38.462816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.462307    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"
Sep 18 22:49:38.462816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.462338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb" event={"ID":"784e159c-a2b7-4659-9a1c-dc00a0741516","Type":"ContainerDied","Data":"2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db"}
Sep 18 22:49:38.462816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.462403    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb" event={"ID":"784e159c-a2b7-4659-9a1c-dc00a0741516","Type":"ContainerDied","Data":"dce9ea5f56a9f91d1ddd87a85ca486f4d4ba98c533de12bd9f01e13011751847"}
Sep 18 22:49:38.462816 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.462423    2583 scope.go:117] "RemoveContainer" containerID="2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db"
Sep 18 22:49:38.463904 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.463879    2583 generic.go:356] "Generic (PLEG): container finished" podID="dcb7ab72-26a2-4a70-bbca-d2204c00a845" containerID="34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67" exitCode=0
Sep 18 22:49:38.463990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.463917    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"
Sep 18 22:49:38.463990 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.463969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8" event={"ID":"dcb7ab72-26a2-4a70-bbca-d2204c00a845","Type":"ContainerDied","Data":"34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67"}
Sep 18 22:49:38.464100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.464039    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8" event={"ID":"dcb7ab72-26a2-4a70-bbca-d2204c00a845","Type":"ContainerDied","Data":"58f0334309f802f8a5eee1b65fd61c3d8c92f2b63364c51c191fc1a77a924347"}
Sep 18 22:49:38.465681 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.465658    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"
Sep 18 22:49:38.465808 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.465660    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz" event={"ID":"b39fa293-85fc-4979-9ad6-e0da0c645e26","Type":"ContainerDied","Data":"4e85d1ae0cc80dd3d21f786665d68d8558e40d0d8583cee0d4fcb96fc0640395"}
Sep 18 22:49:38.473946 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.473927    2583 scope.go:117] "RemoveContainer" containerID="2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db"
Sep 18 22:49:38.474275 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:49:38.474246    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db\": container with ID starting with 2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db not found: ID does not exist" containerID="2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db"
Sep 18 22:49:38.474379 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.474281    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db"} err="failed to get container status \"2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db\": rpc error: code = NotFound desc = could not find container \"2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db\": container with ID starting with 2e34cfc3a5f55ca1bc232f675533f6558e4d0e5d80377ab2fdb4f235ad0be8db not found: ID does not exist"
Sep 18 22:49:38.474379 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.474305    2583 scope.go:117] "RemoveContainer" containerID="34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67"
Sep 18 22:49:38.482920 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.482903    2583 scope.go:117] "RemoveContainer" containerID="34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67"
Sep 18 22:49:38.483194 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:49:38.483177    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67\": container with ID starting with 34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67 not found: ID does not exist" containerID="34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67"
Sep 18 22:49:38.483243 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.483200    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67"} err="failed to get container status \"34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67\": rpc error: code = NotFound desc = could not find container \"34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67\": container with ID starting with 34d17972f43662efaab1b99004a72bde7366e5502486f1efb4788442c0aebd67 not found: ID does not exist"
Sep 18 22:49:38.483243 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.483216    2583 scope.go:117] "RemoveContainer" containerID="ff984756395e84e77e6821e1ab490f61fc91a5b983b1c97191f1aec863422246"
Sep 18 22:49:38.514178 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.514121    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"]
Sep 18 22:49:38.529096 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.529066    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e1f79c-69f74b7dfc-7p2rz"]
Sep 18 22:49:38.604307 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.604273    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"]
Sep 18 22:49:38.621909 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.621870    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e1f79c-7977b4459f-pbsz8"]
Sep 18 22:49:38.657958 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.657929    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"]
Sep 18 22:49:38.669362 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:38.669333    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e1f79c-79cd7f5974-4pqqb"]
Sep 18 22:49:39.683437 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:39.683350    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784e159c-a2b7-4659-9a1c-dc00a0741516" path="/var/lib/kubelet/pods/784e159c-a2b7-4659-9a1c-dc00a0741516/volumes"
Sep 18 22:49:39.683829 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:39.683813    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b39fa293-85fc-4979-9ad6-e0da0c645e26" path="/var/lib/kubelet/pods/b39fa293-85fc-4979-9ad6-e0da0c645e26/volumes"
Sep 18 22:49:39.684185 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:49:39.684172    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcb7ab72-26a2-4a70-bbca-d2204c00a845" path="/var/lib/kubelet/pods/dcb7ab72-26a2-4a70-bbca-d2204c00a845/volumes"
Sep 18 22:51:09.865314 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:09.865219    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"]
Sep 18 22:51:09.868338 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:09.868312    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-d6c0ac-29829525-2xrxh"]
Sep 18 22:51:11.682983 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:11.682945    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f247812-7caf-4b55-b99d-a152dc737f24" path="/var/lib/kubelet/pods/5f247812-7caf-4b55-b99d-a152dc737f24/volumes"
Sep 18 22:51:27.816969 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:27.816933    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"]
Sep 18 22:51:27.817564 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:27.817168    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" podUID="db4fad39-1998-4a2c-85c8-545da2eafb75" containerName="maas-api" containerID="cri-o://c3524555b308308e1e84cb9142e3ba9cfaaf340ffcb19d6df5c0f56d0ffe5e8e" gracePeriod=30
Sep 18 22:51:27.871266 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:27.871225    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"]
Sep 18 22:51:27.871527 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:27.871494    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8" podUID="a8e05e15-28de-404b-8e84-e9ad5fbebcd1" containerName="payload-processing" containerID="cri-o://33109b36579c263ae7d714f1552ac0465d5353919081032d6b642a020016d9ae" gracePeriod=30
Sep 18 22:51:27.879686 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:27.879647    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"]
Sep 18 22:51:27.879927 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:27.879897    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl" podUID="f5342023-b122-43e8-aa1d-00c093d4ae5b" containerName="payload-pre-processing" containerID="cri-o://fde3b8b0056735de7e40c60bff8910784db6005c6dc8ba75c4d6faab4ffb97b4" gracePeriod=30
Sep 18 22:51:28.070230 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.070143    2583 generic.go:356] "Generic (PLEG): container finished" podID="f5342023-b122-43e8-aa1d-00c093d4ae5b" containerID="fde3b8b0056735de7e40c60bff8910784db6005c6dc8ba75c4d6faab4ffb97b4" exitCode=0
Sep 18 22:51:28.070230 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.070195    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl" event={"ID":"f5342023-b122-43e8-aa1d-00c093d4ae5b","Type":"ContainerDied","Data":"fde3b8b0056735de7e40c60bff8910784db6005c6dc8ba75c4d6faab4ffb97b4"}
Sep 18 22:51:28.072454 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.072372    2583 generic.go:356] "Generic (PLEG): container finished" podID="db4fad39-1998-4a2c-85c8-545da2eafb75" containerID="c3524555b308308e1e84cb9142e3ba9cfaaf340ffcb19d6df5c0f56d0ffe5e8e" exitCode=0
Sep 18 22:51:28.072605 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.072530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" event={"ID":"db4fad39-1998-4a2c-85c8-545da2eafb75","Type":"ContainerDied","Data":"c3524555b308308e1e84cb9142e3ba9cfaaf340ffcb19d6df5c0f56d0ffe5e8e"}
Sep 18 22:51:28.074778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.074635    2583 generic.go:356] "Generic (PLEG): container finished" podID="a8e05e15-28de-404b-8e84-e9ad5fbebcd1" containerID="33109b36579c263ae7d714f1552ac0465d5353919081032d6b642a020016d9ae" exitCode=0
Sep 18 22:51:28.074778 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.074715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8" event={"ID":"a8e05e15-28de-404b-8e84-e9ad5fbebcd1","Type":"ContainerDied","Data":"33109b36579c263ae7d714f1552ac0465d5353919081032d6b642a020016d9ae"}
Sep 18 22:51:28.131547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.131517    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:51:28.168355 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.168326    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:51:28.176970 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.176942    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:51:28.255203 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255169    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls\") pod \"db4fad39-1998-4a2c-85c8-545da2eafb75\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") "
Sep 18 22:51:28.255378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255257    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbg7f\" (UniqueName: \"kubernetes.io/projected/db4fad39-1998-4a2c-85c8-545da2eafb75-kube-api-access-kbg7f\") pod \"db4fad39-1998-4a2c-85c8-545da2eafb75\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") "
Sep 18 22:51:28.255378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255304    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tzsj\" (UniqueName: \"kubernetes.io/projected/f5342023-b122-43e8-aa1d-00c093d4ae5b-kube-api-access-9tzsj\") pod \"f5342023-b122-43e8-aa1d-00c093d4ae5b\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") "
Sep 18 22:51:28.255378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255334    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5342023-b122-43e8-aa1d-00c093d4ae5b-plugins-config-volume\") pod \"f5342023-b122-43e8-aa1d-00c093d4ae5b\" (UID: \"f5342023-b122-43e8-aa1d-00c093d4ae5b\") "
Sep 18 22:51:28.255378 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255379    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-plugins-config-volume\") pod \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") "
Sep 18 22:51:28.255644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255442    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-metrics-certs\") pod \"db4fad39-1998-4a2c-85c8-545da2eafb75\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") "
Sep 18 22:51:28.255644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255493    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4fad39-1998-4a2c-85c8-545da2eafb75-openshift-service-ca\") pod \"db4fad39-1998-4a2c-85c8-545da2eafb75\" (UID: \"db4fad39-1998-4a2c-85c8-545da2eafb75\") "
Sep 18 22:51:28.255644 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255531    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlp6d\" (UniqueName: \"kubernetes.io/projected/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-kube-api-access-dlp6d\") pod \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\" (UID: \"a8e05e15-28de-404b-8e84-e9ad5fbebcd1\") "
Sep 18 22:51:28.255885 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255792    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5342023-b122-43e8-aa1d-00c093d4ae5b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5342023-b122-43e8-aa1d-00c093d4ae5b" (UID: "f5342023-b122-43e8-aa1d-00c093d4ae5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:51:28.256050 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.255987    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8e05e15-28de-404b-8e84-e9ad5fbebcd1" (UID: "a8e05e15-28de-404b-8e84-e9ad5fbebcd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:51:28.256173 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.256151    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db4fad39-1998-4a2c-85c8-545da2eafb75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db4fad39-1998-4a2c-85c8-545da2eafb75" (UID: "db4fad39-1998-4a2c-85c8-545da2eafb75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:51:28.257893 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.257867    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db4fad39-1998-4a2c-85c8-545da2eafb75" (UID: "db4fad39-1998-4a2c-85c8-545da2eafb75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:51:28.257893 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.257878    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4fad39-1998-4a2c-85c8-545da2eafb75-kube-api-access-kbg7f" (OuterVolumeSpecName: "kube-api-access-kbg7f") pod "db4fad39-1998-4a2c-85c8-545da2eafb75" (UID: "db4fad39-1998-4a2c-85c8-545da2eafb75"). InnerVolumeSpecName "kube-api-access-kbg7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:51:28.258046 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.257904    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5342023-b122-43e8-aa1d-00c093d4ae5b-kube-api-access-9tzsj" (OuterVolumeSpecName: "kube-api-access-9tzsj") pod "f5342023-b122-43e8-aa1d-00c093d4ae5b" (UID: "f5342023-b122-43e8-aa1d-00c093d4ae5b"). InnerVolumeSpecName "kube-api-access-9tzsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:51:28.258046 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.257995    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "db4fad39-1998-4a2c-85c8-545da2eafb75" (UID: "db4fad39-1998-4a2c-85c8-545da2eafb75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:51:28.258309 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.258279    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-kube-api-access-dlp6d" (OuterVolumeSpecName: "kube-api-access-dlp6d") pod "a8e05e15-28de-404b-8e84-e9ad5fbebcd1" (UID: "a8e05e15-28de-404b-8e84-e9ad5fbebcd1"). InnerVolumeSpecName "kube-api-access-dlp6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356742    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbg7f\" (UniqueName: \"kubernetes.io/projected/db4fad39-1998-4a2c-85c8-545da2eafb75-kube-api-access-kbg7f\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356772    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tzsj\" (UniqueName: \"kubernetes.io/projected/f5342023-b122-43e8-aa1d-00c093d4ae5b-kube-api-access-9tzsj\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356783    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5342023-b122-43e8-aa1d-00c093d4ae5b-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356793    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-plugins-config-volume\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356804    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356812    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4fad39-1998-4a2c-85c8-545da2eafb75-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356820    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlp6d\" (UniqueName: \"kubernetes.io/projected/a8e05e15-28de-404b-8e84-e9ad5fbebcd1-kube-api-access-dlp6d\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:28.356824 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:28.356828    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4fad39-1998-4a2c-85c8-545da2eafb75-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:29.081691 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.081602    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"
Sep 18 22:51:29.081691 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.081599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl" event={"ID":"f5342023-b122-43e8-aa1d-00c093d4ae5b","Type":"ContainerDied","Data":"bfe334fff2fe1a24613061a7acf3db3527688e12ca6db3bc2dee367dcbe0d99e"}
Sep 18 22:51:29.082188 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.081724    2583 scope.go:117] "RemoveContainer" containerID="fde3b8b0056735de7e40c60bff8910784db6005c6dc8ba75c4d6faab4ffb97b4"
Sep 18 22:51:29.083170 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.083117    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"
Sep 18 22:51:29.083170 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.083138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn" event={"ID":"db4fad39-1998-4a2c-85c8-545da2eafb75","Type":"ContainerDied","Data":"bd6b3af419b042cd1089a133b83cc781dc4f455614b8723ee42ece63a20f3613"}
Sep 18 22:51:29.084763 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.084737    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8" event={"ID":"a8e05e15-28de-404b-8e84-e9ad5fbebcd1","Type":"ContainerDied","Data":"78e4a8194193e8acca01185a0f72bf4dac3eaa64d03fc082feeac1543f9c6291"}
Sep 18 22:51:29.084903 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.084749    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"
Sep 18 22:51:29.095414 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.094653    2583 scope.go:117] "RemoveContainer" containerID="c3524555b308308e1e84cb9142e3ba9cfaaf340ffcb19d6df5c0f56d0ffe5e8e"
Sep 18 22:51:29.107701 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.107680    2583 scope.go:117] "RemoveContainer" containerID="33109b36579c263ae7d714f1552ac0465d5353919081032d6b642a020016d9ae"
Sep 18 22:51:29.118487 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.118459    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"]
Sep 18 22:51:29.123005 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.122979    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a91c21-7689d9b9cd-vvsgl"]
Sep 18 22:51:29.134911 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.134891    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"]
Sep 18 22:51:29.140143 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.140119    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a91c21-5b997cb49c-m94mn"]
Sep 18 22:51:29.153240 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.153213    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"]
Sep 18 22:51:29.156326 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.156305    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a91c21-6987df6979-8tsq8"]
Sep 18 22:51:29.683168 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.683135    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8e05e15-28de-404b-8e84-e9ad5fbebcd1" path="/var/lib/kubelet/pods/a8e05e15-28de-404b-8e84-e9ad5fbebcd1/volumes"
Sep 18 22:51:29.683603 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.683589    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db4fad39-1998-4a2c-85c8-545da2eafb75" path="/var/lib/kubelet/pods/db4fad39-1998-4a2c-85c8-545da2eafb75/volumes"
Sep 18 22:51:29.683948 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:29.683934    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5342023-b122-43e8-aa1d-00c093d4ae5b" path="/var/lib/kubelet/pods/f5342023-b122-43e8-aa1d-00c093d4ae5b/volumes"
Sep 18 22:51:37.511089 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.511053    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8567457bbb-5ghhv"]
Sep 18 22:51:37.511523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.511272    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8567457bbb-5ghhv" podUID="d6fe8c04-97d5-4a24-9588-5dde268a6a79" containerName="manager" containerID="cri-o://89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4" gracePeriod=30
Sep 18 22:51:37.768199 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.768135    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:51:37.943768 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.943737    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-webhook-cert\") pod \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") "
Sep 18 22:51:37.943936 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.943809    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b49kl\" (UniqueName: \"kubernetes.io/projected/d6fe8c04-97d5-4a24-9588-5dde268a6a79-kube-api-access-b49kl\") pod \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") "
Sep 18 22:51:37.943936 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.943854    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-metrics-certs\") pod \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\" (UID: \"d6fe8c04-97d5-4a24-9588-5dde268a6a79\") "
Sep 18 22:51:37.945854 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.945818    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d6fe8c04-97d5-4a24-9588-5dde268a6a79" (UID: "d6fe8c04-97d5-4a24-9588-5dde268a6a79"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:51:37.945967 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.945891    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6fe8c04-97d5-4a24-9588-5dde268a6a79" (UID: "d6fe8c04-97d5-4a24-9588-5dde268a6a79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:51:37.946036 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:37.946021    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6fe8c04-97d5-4a24-9588-5dde268a6a79-kube-api-access-b49kl" (OuterVolumeSpecName: "kube-api-access-b49kl") pod "d6fe8c04-97d5-4a24-9588-5dde268a6a79" (UID: "d6fe8c04-97d5-4a24-9588-5dde268a6a79"). InnerVolumeSpecName "kube-api-access-b49kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:51:38.045067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.044985    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:38.045067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.045010    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6fe8c04-97d5-4a24-9588-5dde268a6a79-webhook-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:38.045067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.045022    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b49kl\" (UniqueName: \"kubernetes.io/projected/d6fe8c04-97d5-4a24-9588-5dde268a6a79-kube-api-access-b49kl\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:51:38.136341 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.136302    2583 generic.go:356] "Generic (PLEG): container finished" podID="d6fe8c04-97d5-4a24-9588-5dde268a6a79" containerID="89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4" exitCode=0
Sep 18 22:51:38.136532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.136350    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8567457bbb-5ghhv"
Sep 18 22:51:38.136532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.136350    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8567457bbb-5ghhv" event={"ID":"d6fe8c04-97d5-4a24-9588-5dde268a6a79","Type":"ContainerDied","Data":"89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4"}
Sep 18 22:51:38.136532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.136484    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8567457bbb-5ghhv" event={"ID":"d6fe8c04-97d5-4a24-9588-5dde268a6a79","Type":"ContainerDied","Data":"88a3417e3b4bf65f53573d23a1ce8348c93938332ddec6a417c2f23012c16cdf"}
Sep 18 22:51:38.136532 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.136512    2583 scope.go:117] "RemoveContainer" containerID="89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4"
Sep 18 22:51:38.147274 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.147256    2583 scope.go:117] "RemoveContainer" containerID="89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4"
Sep 18 22:51:38.147584 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:51:38.147559    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4\": container with ID starting with 89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4 not found: ID does not exist" containerID="89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4"
Sep 18 22:51:38.147665 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.147590    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4"} err="failed to get container status \"89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4\": rpc error: code = NotFound desc = could not find container \"89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4\": container with ID starting with 89ee0bc073aefa2414e00d289229eae2da9b8b6aae00af0d430754e8bb1c89e4 not found: ID does not exist"
Sep 18 22:51:38.160679 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.160658    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8567457bbb-5ghhv"]
Sep 18 22:51:38.164092 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:38.164069    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8567457bbb-5ghhv"]
Sep 18 22:51:39.682675 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:39.682640    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6fe8c04-97d5-4a24-9588-5dde268a6a79" path="/var/lib/kubelet/pods/d6fe8c04-97d5-4a24-9588-5dde268a6a79/volumes"
Sep 18 22:51:57.785088 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:57.785060    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:51:57.789553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:57.789532    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:51:58.045195 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:51:58.045118    2583 scope.go:117] "RemoveContainer" containerID="c6d0bbdaa735265141831c2e8134f90007a4e175400da8f0913c76c655ba84f7"
Sep 18 22:56:12.586937 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:12.586895    2583 controller.go:195] "Failed to update lease" err="Put \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-49.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 18 22:56:17.641183 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:17.641134    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-18T22:56:07Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-18T22:56:07Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-18T22:56:07Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-18T22:56:07Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-132-49.ec2.internal\": Patch \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-49.ec2.internal/status?timeout=10s\": context deadline exceeded"
Sep 18 22:56:22.587920 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:22.587886    2583 controller.go:195] "Failed to update lease" err="Put \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-49.ec2.internal?timeout=10s\": context deadline exceeded"
Sep 18 22:56:27.642260 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:27.642224    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-49.ec2.internal\": Get \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-49.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 18 22:56:32.588139 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:32.588079    2583 controller.go:195] "Failed to update lease" err="Put \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-49.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 18 22:56:37.642624 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:37.642591    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-49.ec2.internal\": Get \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-49.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 18 22:56:39.539795 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539745    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xd66v\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539795    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zwjfq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539814    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539805    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539836    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539835    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f9cpv\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539866    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539871    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539881    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539893    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539902    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"default-dockercfg-s9rh9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539930    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539932    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539957    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539963    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.539976    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540024    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540053    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.540289 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540074    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-95pvs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:56:39.540152    2583 controller.go:195] "Failed to update lease" err="Put \"https://a627019127b114ed0af02f697a598be6-bfa5ca75b4963a39.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-49.ec2.internal?timeout=10s\": http2: client connection lost"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540178    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540209    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540248    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540259    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540269    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540285    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540299    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dq7t2\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540338    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540340    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540351    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-867hh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540360    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540374    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540381    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540428    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540433    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540445    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.541245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540467    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540483    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540486    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540501    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540507    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lcpmb\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540506    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540538    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540564    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-nfrnv\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540574    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540584    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540589    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540597    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540613    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540621    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540645    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540663    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540669    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540618    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4t99f\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.542167 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540676    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540671    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540676    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540702    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540719    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540726    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gtntj\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540729    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-m8757\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540721    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540742    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540778    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4rksw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540779    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540809    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540850    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540859    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540865    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540868    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540879    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"default-dockercfg-dvgzf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540853    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540890    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540891    2583 reflector.go:571] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540913    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rrh9w\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540916    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jhrmn\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540923    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v9tgp\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540924    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540914    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9mm6r\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540939    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540951    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-95qgb\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540955    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540952    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540971    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540975    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.540986    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541001    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541008    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541020    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541027    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.543874 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541030    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lv824\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541047    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541050    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541060    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541070    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541080    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9km2v\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541084    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541102    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541110    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541112    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5v8j7\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541114    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541135    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541154    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-825pr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541158    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541162    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541157    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541126    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541184    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.544643 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541193    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541197    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541186    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541226    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541233    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbrk8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541236    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541244    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541252    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541258    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541267    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-dockercfg-jg4r5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541269    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541272    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4p6sp\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541289    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t5blr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541295    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541289    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541309    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541315    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541314    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545234 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541326    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541336    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gkpcg\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541349    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7xc5k\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541363    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541373    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541403    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541363    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-d2mfekvbrpirl\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541378    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541422    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541336    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6vdvw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:39.545801 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:39.541422    2583 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Sep 18 22:56:40.351745 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.351716    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 18 22:56:40.352610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.352592    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d2mfekvbrpirl\""
Sep 18 22:56:40.357767 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.357751    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.360205 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.360190    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 22:56:40.362736 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.362721    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.392912 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.392886    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.395514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.395500    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9km2v\""
Sep 18 22:56:40.410000 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.409981    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 22:56:40.417180 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.417162    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.429740 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.429717    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.431232 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.431216    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-95qgb\""
Sep 18 22:56:40.432309 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.432292    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.432360 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.432347    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 22:56:40.433905 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.433889    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 22:56:40.434559 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.434545    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9mm6r\""
Sep 18 22:56:40.438597 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.438583    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 22:56:40.455829 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.455809    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 22:56:40.478840 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.478818    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 22:56:40.483291 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.483278    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbrk8\""
Sep 18 22:56:40.486417 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.486400    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 22:56:40.488197 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.488183    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 22:56:40.499548 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.499532    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.515663 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.515646    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 22:56:40.520429 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.520411    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jhrmn\""
Sep 18 22:56:40.548138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.548114    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 18 22:56:40.552553 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.552538    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 22:56:40.573066 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.573036    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:56:40.576366 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.576348    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.577970 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.577957    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 22:56:40.579256 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.579239    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.579547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.579527    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:56:40.588182 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.588167    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 18 22:56:40.599171 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.599150    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.607091 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.607040    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 22:56:40.611255 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.611238    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jg4r5\""
Sep 18 22:56:40.620709 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.620692    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-867hh\""
Sep 18 22:56:40.633608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.633590    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 22:56:40.635346 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.635329    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.643986 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.643967    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4rksw\""
Sep 18 22:56:40.651478 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.651459    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.651557 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.651467    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.655983 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.655967    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.677542 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.677522    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.684038 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.684020    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 22:56:40.693315 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.693299    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v9tgp\""
Sep 18 22:56:40.696144 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.696128    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.698048 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.698034    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 22:56:40.699078 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.699063    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 22:56:40.704280 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.704266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 22:56:40.712273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.712259    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.723226 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.723203    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.728486 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.728467    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dq7t2\""
Sep 18 22:56:40.732787 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.732765    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 22:56:40.733894 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.733880    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.738280 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.738263    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.746928 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.746901    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 22:56:40.747048 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.746902    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 18 22:56:40.747048 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.746902    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 22:56:40.764027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.764002    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 22:56:40.772693 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.772671    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 22:56:40.782292 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.782272    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.785727 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.785711    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xd66v\""
Sep 18 22:56:40.788497 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.788482    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.801064 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.801046    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lcpmb\""
Sep 18 22:56:40.801127 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.801117    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.801858 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.801844    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zwjfq\""
Sep 18 22:56:40.802366 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.802351    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.820937 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.820915    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t5blr\""
Sep 18 22:56:40.820937 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.820934    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.829420 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.829382    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 22:56:40.837807 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.837790    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 22:56:40.848245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.848227    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.849307 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.849292    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.851207 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.851190    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 22:56:40.855514 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.855500    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 22:56:40.858347 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.858301    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 22:56:40.876940 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.876918    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.881200 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.881186    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 22:56:40.882528 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.882512    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 22:56:40.898867 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.898851    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s9rh9\""
Sep 18 22:56:40.900435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.900417    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.900507 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.900425    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 22:56:40.904858 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.904844    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.914163 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.914146    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.915626 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.915614    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 22:56:40.919910 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.919895    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 18 22:56:40.920697 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.920682    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 22:56:40.922303 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.922288    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 22:56:40.932588 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.932571    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 22:56:40.941220 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.941200    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.942138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.942124    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lv824\""
Sep 18 22:56:40.945146 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.945129    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gkpcg\""
Sep 18 22:56:40.951258 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.951240    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 18 22:56:40.953026 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.953012    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 18 22:56:40.958271 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.958254    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4t99f\""
Sep 18 22:56:40.958342 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.958280    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 18 22:56:40.963523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.963501    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 22:56:40.963656 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.963530    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 18 22:56:40.965287 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.965263    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:40.973825 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.973799    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 22:56:40.974106 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.974090    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-95pvs\""
Sep 18 22:56:40.986625 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:40.986607    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 22:56:41.000572 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.000551    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rrh9w\""
Sep 18 22:56:41.002515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.002496    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5v8j7\""
Sep 18 22:56:41.002610 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.002543    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 22:56:41.014506 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.014483    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 22:56:41.015495 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.015474    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7xc5k\""
Sep 18 22:56:41.017520 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.017505    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 22:56:41.024145 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.024123    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 22:56:41.028502 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.028486    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:56:41.028600 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.028574    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 22:56:41.028671 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.028574    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 22:56:41.028722 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.028693    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 18 22:56:41.037046 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.037029    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 22:56:41.038214 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.038199    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 22:56:41.042555 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.042540    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 22:56:41.049651 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.049628    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-nfrnv\""
Sep 18 22:56:41.051359 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.051341    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 22:56:41.051435 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.051379    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 22:56:41.054066 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.054052    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dvgzf\""
Sep 18 22:56:41.058186 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.058172    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 18 22:56:41.060847 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.060831    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-m8757\""
Sep 18 22:56:41.074067 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.074048    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f9cpv\""
Sep 18 22:56:41.075805 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.075790    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 22:56:41.078523 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.078507    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 22:56:41.099250 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.099234    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4p6sp\""
Sep 18 22:56:41.100742 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.100728    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-825pr\""
Sep 18 22:56:41.102213 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.102197    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 22:56:41.109515 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.109465    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 22:56:41.112499 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.112483    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 22:56:41.121984 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.121966    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 22:56:41.122756 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.122743    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gtntj\""
Sep 18 22:56:41.126953 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.126938    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 22:56:41.133334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.133319    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 22:56:41.141055 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.141038    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6vdvw\""
Sep 18 22:56:41.154119 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:41.154093    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 22:56:57.819690 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:57.819663    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:56:57.825237 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:57.825216    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:56:58.073793 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.073712    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"]
Sep 18 22:56:58.074045 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074033    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4fad39-1998-4a2c-85c8-545da2eafb75" containerName="maas-api"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074047    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4fad39-1998-4a2c-85c8-545da2eafb75" containerName="maas-api"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074054    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb6a888d-59a3-4762-b0aa-3814027a3b5b" containerName="payload-processing"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074061    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb6a888d-59a3-4762-b0aa-3814027a3b5b" containerName="payload-processing"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074069    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8e05e15-28de-404b-8e84-e9ad5fbebcd1" containerName="payload-processing"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074075    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8e05e15-28de-404b-8e84-e9ad5fbebcd1" containerName="payload-processing"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074084    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcb7ab72-26a2-4a70-bbca-d2204c00a845" containerName="payload-processing"
Sep 18 22:56:58.074087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074090    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcb7ab72-26a2-4a70-bbca-d2204c00a845" containerName="payload-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074099    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6fe8c04-97d5-4a24-9588-5dde268a6a79" containerName="manager"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074104    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6fe8c04-97d5-4a24-9588-5dde268a6a79" containerName="manager"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074110    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b39fa293-85fc-4979-9ad6-e0da0c645e26" containerName="maas-api"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074115    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b39fa293-85fc-4979-9ad6-e0da0c645e26" containerName="maas-api"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074125    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5342023-b122-43e8-aa1d-00c093d4ae5b" containerName="payload-pre-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074131    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5342023-b122-43e8-aa1d-00c093d4ae5b" containerName="payload-pre-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074140    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784e159c-a2b7-4659-9a1c-dc00a0741516" containerName="payload-pre-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074145    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="784e159c-a2b7-4659-9a1c-dc00a0741516" containerName="payload-pre-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074150    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9674939a-dc7c-4eba-836b-f370f435fb91" containerName="maas-api"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074155    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9674939a-dc7c-4eba-836b-f370f435fb91" containerName="maas-api"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074216    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcb7ab72-26a2-4a70-bbca-d2204c00a845" containerName="payload-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074223    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb6a888d-59a3-4762-b0aa-3814027a3b5b" containerName="payload-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074232    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="784e159c-a2b7-4659-9a1c-dc00a0741516" containerName="payload-pre-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074236    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4fad39-1998-4a2c-85c8-545da2eafb75" containerName="maas-api"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074243    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6fe8c04-97d5-4a24-9588-5dde268a6a79" containerName="manager"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074250    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9674939a-dc7c-4eba-836b-f370f435fb91" containerName="maas-api"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074256    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5342023-b122-43e8-aa1d-00c093d4ae5b" containerName="payload-pre-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074262    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8e05e15-28de-404b-8e84-e9ad5fbebcd1" containerName="payload-processing"
Sep 18 22:56:58.074301 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.074267    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b39fa293-85fc-4979-9ad6-e0da0c645e26" containerName="maas-api"
Sep 18 22:56:58.077200 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.077184    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.080771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.080749    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 22:56:58.080771 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.080760    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 22:56:58.080977 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.080764    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 22:56:58.080977 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.080787    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7xxrx\""
Sep 18 22:56:58.080977 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.080794    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 22:56:58.088623 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.088601    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"]
Sep 18 22:56:58.102423 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.102379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-metrics-certs\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.102570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.102462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0381c-d070-4043-ab47-6f92ea0c0ee9-openshift-service-ca\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.102570 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.102549    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-maas-api-tls\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.102693 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.102582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s5fv\" (UniqueName: \"kubernetes.io/projected/abf0381c-d070-4043-ab47-6f92ea0c0ee9-kube-api-access-8s5fv\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.203273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.203239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-metrics-certs\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.203483 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.203281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0381c-d070-4043-ab47-6f92ea0c0ee9-openshift-service-ca\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.203483 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.203340    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-maas-api-tls\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.203483 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.203362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s5fv\" (UniqueName: \"kubernetes.io/projected/abf0381c-d070-4043-ab47-6f92ea0c0ee9-kube-api-access-8s5fv\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.204074 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.204048    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0381c-d070-4043-ab47-6f92ea0c0ee9-openshift-service-ca\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.205881 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.205852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-metrics-certs\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.205989 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.205849    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-maas-api-tls\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.210651 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.210631    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s5fv\" (UniqueName: \"kubernetes.io/projected/abf0381c-d070-4043-ab47-6f92ea0c0ee9-kube-api-access-8s5fv\") pod \"maas-api-5598d8c9fb-6sq5g\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.388735 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.388703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:58.682775 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.682749    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"]
Sep 18 22:56:58.684686 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:56:58.684660    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf0381c_d070_4043_ab47_6f92ea0c0ee9.slice/crio-7212f2b6fff0356a34effcae467501a7e2bcff2a65236fc4d504a9956eab9c18 WatchSource:0}: Error finding container 7212f2b6fff0356a34effcae467501a7e2bcff2a65236fc4d504a9956eab9c18: Status 404 returned error can't find the container with id 7212f2b6fff0356a34effcae467501a7e2bcff2a65236fc4d504a9956eab9c18
Sep 18 22:56:58.686153 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.686133    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 22:56:58.775027 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:58.774988    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g" event={"ID":"abf0381c-d070-4043-ab47-6f92ea0c0ee9","Type":"ContainerStarted","Data":"7212f2b6fff0356a34effcae467501a7e2bcff2a65236fc4d504a9956eab9c18"}
Sep 18 22:56:59.784248 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:59.784208    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g" event={"ID":"abf0381c-d070-4043-ab47-6f92ea0c0ee9","Type":"ContainerStarted","Data":"6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738"}
Sep 18 22:56:59.784658 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:59.784344    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:56:59.808354 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:56:59.808304    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g" podStartSLOduration=1.363065437 podStartE2EDuration="1.808288771s" totalImagesPullingTime="445.223334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 22:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:56:59.806889833 +0000 UTC m=+2404.167406705" watchObservedRunningTime="2026-09-18 22:56:59.808288771 +0000 UTC m=+2404.168805628"
Sep 18 22:57:05.773776 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:05.773746    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"]
Sep 18 22:57:05.774210 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:05.773912    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g" podUID="abf0381c-d070-4043-ab47-6f92ea0c0ee9" containerName="maas-api" containerID="cri-o://6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738" gracePeriod=30
Sep 18 22:57:05.779751 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:05.779727    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:57:06.024202 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.024132    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:57:06.066628 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.066593    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-metrics-certs\") pod \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") "
Sep 18 22:57:06.066791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.066638    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s5fv\" (UniqueName: \"kubernetes.io/projected/abf0381c-d070-4043-ab47-6f92ea0c0ee9-kube-api-access-8s5fv\") pod \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") "
Sep 18 22:57:06.066791 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.066696    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-maas-api-tls\") pod \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") "
Sep 18 22:57:06.066878 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.066792    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0381c-d070-4043-ab47-6f92ea0c0ee9-openshift-service-ca\") pod \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\" (UID: \"abf0381c-d070-4043-ab47-6f92ea0c0ee9\") "
Sep 18 22:57:06.067226 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.067186    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf0381c-d070-4043-ab47-6f92ea0c0ee9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abf0381c-d070-4043-ab47-6f92ea0c0ee9" (UID: "abf0381c-d070-4043-ab47-6f92ea0c0ee9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 22:57:06.068864 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.068832    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "abf0381c-d070-4043-ab47-6f92ea0c0ee9" (UID: "abf0381c-d070-4043-ab47-6f92ea0c0ee9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:57:06.068971 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.068863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abf0381c-d070-4043-ab47-6f92ea0c0ee9" (UID: "abf0381c-d070-4043-ab47-6f92ea0c0ee9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 22:57:06.068971 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.068913    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf0381c-d070-4043-ab47-6f92ea0c0ee9-kube-api-access-8s5fv" (OuterVolumeSpecName: "kube-api-access-8s5fv") pod "abf0381c-d070-4043-ab47-6f92ea0c0ee9" (UID: "abf0381c-d070-4043-ab47-6f92ea0c0ee9"). InnerVolumeSpecName "kube-api-access-8s5fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 22:57:06.168101 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.168057    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:57:06.168101 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.168097    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf0381c-d070-4043-ab47-6f92ea0c0ee9-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:57:06.168101 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.168107    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf0381c-d070-4043-ab47-6f92ea0c0ee9-metrics-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:57:06.168101 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.168116    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s5fv\" (UniqueName: \"kubernetes.io/projected/abf0381c-d070-4043-ab47-6f92ea0c0ee9-kube-api-access-8s5fv\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Sep 18 22:57:06.820827 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.820794    2583 generic.go:356] "Generic (PLEG): container finished" podID="abf0381c-d070-4043-ab47-6f92ea0c0ee9" containerID="6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738" exitCode=0
Sep 18 22:57:06.821270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.820843    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"
Sep 18 22:57:06.821270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.820878    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g" event={"ID":"abf0381c-d070-4043-ab47-6f92ea0c0ee9","Type":"ContainerDied","Data":"6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738"}
Sep 18 22:57:06.821270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.820937    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g" event={"ID":"abf0381c-d070-4043-ab47-6f92ea0c0ee9","Type":"ContainerDied","Data":"7212f2b6fff0356a34effcae467501a7e2bcff2a65236fc4d504a9956eab9c18"}
Sep 18 22:57:06.821270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.820960    2583 scope.go:117] "RemoveContainer" containerID="6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738"
Sep 18 22:57:06.830160 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.830144    2583 scope.go:117] "RemoveContainer" containerID="6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738"
Sep 18 22:57:06.830524 ip-10-0-132-49 kubenswrapper[2583]: E0918 22:57:06.830503    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738\": container with ID starting with 6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738 not found: ID does not exist" containerID="6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738"
Sep 18 22:57:06.830606 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.830535    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738"} err="failed to get container status \"6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738\": rpc error: code = NotFound desc = could not find container \"6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738\": container with ID starting with 6c6b531d6f1f9ec3cfa49de74485ecfe84319467748539fbbcff66d7c2a5b738 not found: ID does not exist"
Sep 18 22:57:06.842922 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.842901    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"]
Sep 18 22:57:06.846730 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:06.846707    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-6sq5g"]
Sep 18 22:57:07.682715 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:07.682679    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf0381c-d070-4043-ab47-6f92ea0c0ee9" path="/var/lib/kubelet/pods/abf0381c-d070-4043-ab47-6f92ea0c0ee9/volumes"
Sep 18 22:57:33.453249 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:33.453221    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m6sss_88b74bf2-c493-4c1b-8532-fcbf9671a6e1/server/0.log"
Sep 18 22:57:33.567633 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:33.567603    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-89gq7_8e117632-92ea-4ec1-9b93-5b98e0088601/manager/0.log"
Sep 18 22:57:33.953047 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:33.953019    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c85dbc8b-z9cgl_83ceed36-4803-45b4-91d6-f1db76358bf0/copy-manifests/0.log"
Sep 18 22:57:33.960142 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:33.960114    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c85dbc8b-z9cgl_83ceed36-4803-45b4-91d6-f1db76358bf0/manager/0.log"
Sep 18 22:57:36.638269 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:36.638242    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z44wc_eef8e44b-6cd2-4943-b7f2-0444ebc9452d/discovery/0.log"
Sep 18 22:57:37.085770 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:37.085694    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-qsknm_904be353-e4a5-48fe-9054-397522c3dd4f/payload-pre-processing/0.log"
Sep 18 22:57:37.202608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:37.202579    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-twfmq_80653214-eddf-4e73-882c-4348f6613d7e/payload-processing/0.log"
Sep 18 22:57:37.768761 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:37.768729    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jnsqt_37224245-8fbc-419b-b588-1b369fc24f1f/main/0.log"
Sep 18 22:57:38.214754 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:38.214719    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xsphf_041109bc-773d-4cda-a03f-679eaec8b8f2/main/0.log"
Sep 18 22:57:47.183985 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:47.183952    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4t6gh_a26f907f-1903-4bf9-b515-62df85b2b6f7/global-pull-secret-syncer/0.log"
Sep 18 22:57:47.381138 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:47.381111    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jst6w_cdacda15-9e2c-4c1e-a142-e5e49b023f95/konnectivity-agent/0.log"
Sep 18 22:57:47.489946 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:47.489841    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-49.ec2.internal_f394ebf0066db702356187a5a575a284/haproxy/0.log"
Sep 18 22:57:54.334198 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:54.334169    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c49ff9684-plf67_f7c00d2e-dc8f-4031-9cd6-1a7a333705cf/metrics-server/0.log"
Sep 18 22:57:54.530757 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:54.530684    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dkqq7_6344af3b-75e5-4207-9e55-39fcf7477705/node-exporter/0.log"
Sep 18 22:57:54.556534 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:54.556510    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dkqq7_6344af3b-75e5-4207-9e55-39fcf7477705/kube-rbac-proxy/0.log"
Sep 18 22:57:54.580933 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:54.580912    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dkqq7_6344af3b-75e5-4207-9e55-39fcf7477705/init-textfile/0.log"
Sep 18 22:57:55.143256 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:55.143223    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5955dd6588-x2nvs_35582a02-2a07-4463-a876-45d92cb4deee/telemeter-client/0.log"
Sep 18 22:57:55.168245 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:55.168218    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5955dd6588-x2nvs_35582a02-2a07-4463-a876-45d92cb4deee/reload/0.log"
Sep 18 22:57:55.198329 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:55.198303    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5955dd6588-x2nvs_35582a02-2a07-4463-a876-45d92cb4deee/kube-rbac-proxy/0.log"
Sep 18 22:57:56.971796 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.971761    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"]
Sep 18 22:57:56.972273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.972233    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf0381c-d070-4043-ab47-6f92ea0c0ee9" containerName="maas-api"
Sep 18 22:57:56.972273 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.972254    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf0381c-d070-4043-ab47-6f92ea0c0ee9" containerName="maas-api"
Sep 18 22:57:56.972381 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.972331    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf0381c-d070-4043-ab47-6f92ea0c0ee9" containerName="maas-api"
Sep 18 22:57:56.975455 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.975437    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:56.977728 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.977707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-558dg\"/\"openshift-service-ca.crt\""
Sep 18 22:57:56.977851 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.977713    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-558dg\"/\"default-dockercfg-nqbcn\""
Sep 18 22:57:56.978132 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.978118    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-558dg\"/\"kube-root-ca.crt\""
Sep 18 22:57:56.988722 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:56.988698    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"]
Sep 18 22:57:57.026585 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.026553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d488\" (UniqueName: \"kubernetes.io/projected/f3524422-323d-4ee4-92db-e9f9ff00ab84-kube-api-access-2d488\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.026760 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.026593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-lib-modules\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.026760 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.026622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-sys\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.026760 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.026679    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-proc\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.026760 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.026705    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-podres\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128194 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-proc\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128194 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-podres\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d488\" (UniqueName: \"kubernetes.io/projected/f3524422-323d-4ee4-92db-e9f9ff00ab84-kube-api-access-2d488\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-lib-modules\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128296    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-proc\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-sys\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128362    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-sys\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128464 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128429    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-podres\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.128710 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.128482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3524422-323d-4ee4-92db-e9f9ff00ab84-lib-modules\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.137608 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.137587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d488\" (UniqueName: \"kubernetes.io/projected/f3524422-323d-4ee4-92db-e9f9ff00ab84-kube-api-access-2d488\") pod \"perf-node-gather-daemonset-qc5tk\" (UID: \"f3524422-323d-4ee4-92db-e9f9ff00ab84\") " pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.285597 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.285509    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:57.302025 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.301996    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/1.log"
Sep 18 22:57:57.308434 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.308410    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-p2rff_e07e370c-b4b7-4974-9297-a24f5647ec3e/console-operator/2.log"
Sep 18 22:57:57.561613 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:57.561585    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"]
Sep 18 22:57:57.564023 ip-10-0-132-49 kubenswrapper[2583]: W0918 22:57:57.563979    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf3524422_323d_4ee4_92db_e9f9ff00ab84.slice/crio-02b88f4dead3b8e4a86fee04dc887d03aab5eb6dfec793b697019819db04a64e WatchSource:0}: Error finding container 02b88f4dead3b8e4a86fee04dc887d03aab5eb6dfec793b697019819db04a64e: Status 404 returned error can't find the container with id 02b88f4dead3b8e4a86fee04dc887d03aab5eb6dfec793b697019819db04a64e
Sep 18 22:57:58.091805 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:58.091775    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk" event={"ID":"f3524422-323d-4ee4-92db-e9f9ff00ab84","Type":"ContainerStarted","Data":"27fe7ef2555ed47c36e9941ad1e789847e4c565b22763cdc234413bf36919b56"}
Sep 18 22:57:58.092223 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:58.091833    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk" event={"ID":"f3524422-323d-4ee4-92db-e9f9ff00ab84","Type":"ContainerStarted","Data":"02b88f4dead3b8e4a86fee04dc887d03aab5eb6dfec793b697019819db04a64e"}
Sep 18 22:57:58.092223 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:58.091863    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:57:58.112898 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:58.112849    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk" podStartSLOduration=2.112833768 podStartE2EDuration="2.112833768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 22:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 22:57:58.110853436 +0000 UTC m=+2462.471370294" watchObservedRunningTime="2026-09-18 22:57:58.112833768 +0000 UTC m=+2462.473350625"
Sep 18 22:57:58.711413 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:57:58.711363    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-qwnls_2dd3cdee-7c07-4315-897a-0755cbe63bf4/volume-data-source-validator/0.log"
Sep 18 22:58:00.085590 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:00.085562    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sqsj6_c843f1a2-c11e-4e96-aa81-998169823479/dns/0.log"
Sep 18 22:58:00.137286 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:00.137262    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sqsj6_c843f1a2-c11e-4e96-aa81-998169823479/kube-rbac-proxy/0.log"
Sep 18 22:58:00.371270 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:00.371193    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r28kg_b39dd412-80b2-4d98-b06a-8c0f1b86a59b/dns-node-resolver/0.log"
Sep 18 22:58:01.023480 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:01.023449    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gdk85_8672980a-1096-46f2-a1ba-d5d1c768e247/node-ca/0.log"
Sep 18 22:58:02.577547 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:02.577509    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z44wc_eef8e44b-6cd2-4943-b7f2-0444ebc9452d/discovery/0.log"
Sep 18 22:58:02.718087 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:02.718054    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-qsknm_904be353-e4a5-48fe-9054-397522c3dd4f/payload-pre-processing/0.log"
Sep 18 22:58:02.755076 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:02.755046    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-twfmq_80653214-eddf-4e73-882c-4348f6613d7e/payload-processing/0.log"
Sep 18 22:58:03.716307 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:03.716273    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hg5t_309bb642-370f-4c18-896e-844cb37b7672/serve-healthcheck-canary/0.log"
Sep 18 22:58:04.106649 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:04.106566    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-558dg/perf-node-gather-daemonset-qc5tk"
Sep 18 22:58:04.249100 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:04.249064    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p75rv_484adadf-5f15-4fe6-b88f-0d6e76b0ae3f/insights-operator/0.log"
Sep 18 22:58:04.249624 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:04.249603    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-p75rv_484adadf-5f15-4fe6-b88f-0d6e76b0ae3f/insights-operator/1.log"
Sep 18 22:58:04.424161 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:04.424131    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rbfcg_53d716ef-46dc-4722-8340-869103572c25/kube-rbac-proxy/0.log"
Sep 18 22:58:04.448034 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:04.448011    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rbfcg_53d716ef-46dc-4722-8340-869103572c25/exporter/0.log"
Sep 18 22:58:04.477634 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:04.477604    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rbfcg_53d716ef-46dc-4722-8340-869103572c25/extractor/0.log"
Sep 18 22:58:06.820334 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:06.820309    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m6sss_88b74bf2-c493-4c1b-8532-fcbf9671a6e1/server/0.log"
Sep 18 22:58:06.914798 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:06.914767    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-89gq7_8e117632-92ea-4ec1-9b93-5b98e0088601/manager/0.log"
Sep 18 22:58:07.179900 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:07.179869    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c85dbc8b-z9cgl_83ceed36-4803-45b4-91d6-f1db76358bf0/manager/0.log"
Sep 18 22:58:07.208752 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:07.208727    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c85dbc8b-z9cgl_83ceed36-4803-45b4-91d6-f1db76358bf0/copy-manifests/0.log"
Sep 18 22:58:07.427376 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:07.427346    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z44wc_eef8e44b-6cd2-4943-b7f2-0444ebc9452d/discovery/0.log"
Sep 18 22:58:07.511545 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:07.511515    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-qsknm_904be353-e4a5-48fe-9054-397522c3dd4f/payload-pre-processing/0.log"
Sep 18 22:58:07.542018 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:07.537364    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-twfmq_80653214-eddf-4e73-882c-4348f6613d7e/payload-processing/0.log"
Sep 18 22:58:08.593406 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:08.593364    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bdd6569c9-bpbw4_d1a354c3-d392-49e6-845d-d18129134b5c/manager/0.log"
Sep 18 22:58:09.600516 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:09.600481    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z44wc_eef8e44b-6cd2-4943-b7f2-0444ebc9452d/discovery/0.log"
Sep 18 22:58:09.697427 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:09.697316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-qsknm_904be353-e4a5-48fe-9054-397522c3dd4f/payload-pre-processing/0.log"
Sep 18 22:58:09.725493 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:09.725458    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-twfmq_80653214-eddf-4e73-882c-4348f6613d7e/payload-processing/0.log"
Sep 18 22:58:15.056939 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:15.056908    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjr2x_0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9/kube-multus-additional-cni-plugins/0.log"
Sep 18 22:58:15.080905 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:15.080880    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjr2x_0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9/egress-router-binary-copy/0.log"
Sep 18 22:58:15.103475 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:15.103445    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjr2x_0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9/cni-plugins/0.log"
Sep 18 22:58:15.130558 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:15.130532    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjr2x_0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9/bond-cni-plugin/0.log"
Sep 18 22:58:15.181738 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:15.181711    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjr2x_0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9/routeoverride-cni/0.log"
Sep 18 22:58:15.261319 ip-10-0-132-49 kubenswrapper[2583]: I0918 22:58:15.261285    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjr2x_0f8bb6e2-d39b-4b10-b2dd-28c84e42c2e9/whereabouts-cni-bincopy/0.log"